-- Logs begin at Sat 2022-09-24 19:04:59 UTC, end at Sat 2022-09-24 20:05:38 UTC. -- Sep 24 19:30:07.098633 dv-cinder-859191-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 24 19:30:07.107132 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 24 19:30:07 2022] *** Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: nodename: dv-cinder-859191-1-iscsi Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: machine: x86_64 Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: clock source: unix Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: pcre jit disabled Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: detected number of CPU cores: 4 Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: current working directory: / Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: detected binary path: /usr/bin/uwsgi-core Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: your processes number limit is 47680 Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: your memory page size is 4096 bytes Sep 24 19:30:07.109171 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: detected max file descriptor number: 2048 Sep 24 19:30:07.109877 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: lock engine: pthread robust mutexes Sep 24 19:30:07.109877 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: thunder lock: enabled Sep 24 19:30:07.109877 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 24 19:30:07.109877 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 24 19:30:07.149347 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: Python main interpreter initialized at 0x5628efce89c0 Sep 24 19:30:07.149347 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: python threads support enabled Sep 24 19:30:07.149347 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: your server socket listen backlog is limited to 100 connections Sep 24 19:30:07.149347 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: your mercy for graceful operations on workers is 90 seconds Sep 24 19:30:07.149702 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: mapped 403077 bytes (393 KB) for 2 cores Sep 24 19:30:07.149782 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: *** Operational MODE: preforking *** Sep 24 19:30:07.150026 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: *** uWSGI is running in multiple interpreter mode *** Sep 24 19:30:07.150026 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: spawned uWSGI master process (pid: 81791) Sep 24 19:30:07.150217 dv-cinder-859191-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 24 19:30:07.155455 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: spawned uWSGI worker 1 (pid: 81801, cores: 1) Sep 24 19:30:07.155455 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: spawned uWSGI worker 2 (pid: 81802, cores: 1) Sep 24 19:30:07.155455 dv-cinder-859191-1-iscsi devstack@keystone.service[81791]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 24 19:30:11.460521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 19:30:11.460521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: from cryptography.utils import int_from_bytes Sep 24 19:30:11.461300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 19:30:11.461300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: from cryptography.utils import int_from_bytes Sep 24 19:30:11.463693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 19:30:11.463693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: from cryptography.utils import int_from_bytes Sep 24 19:30:11.464137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 19:30:11.464137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: from cryptography.utils import int_from_bytes Sep 24 19:30:11.511451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 19:30:11.511657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 19:30:11.511848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] command line args: [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 19:30:11.512119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 19:30:11.512295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ================================================================================ {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 19:30:11.512547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] admin_token = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.512792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] config_dir = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.512978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.513158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] config_source = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.513335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] debug = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.513531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.513774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 19:30:11.513774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 19:30:11.513906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.513971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] command line args: [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 19:30:11.514113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.514198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 19:30:11.514306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.514409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ================================================================================ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 19:30:11.514549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.514655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] admin_token = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.514778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.514896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config_dir = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_config_append = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config_source = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_dir = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] debug = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_file = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.515987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_config_append = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.516982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_dir = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.517701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.518888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] publish_errors = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.519973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.520931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] standard_threads = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] use_journal = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] publish_errors = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.521889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] use_json = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] use_stderr = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] use_syslog = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.522730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.522891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.522974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.523128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] standard_threads = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.523212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.523364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.523464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.523638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.523773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.523857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.523931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.524008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_journal = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.524110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.524199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_json = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.524306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_stderr = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.524386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.524482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_syslog = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.524581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.524716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 19:30:11.524828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.524977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.525974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.526953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.527946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.528962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.529927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.530881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.531843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.532997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.533150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.533230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.533905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.534960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.535919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.536988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.537889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.538975 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.539947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.540929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.541968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.542895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.543957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.544991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.545902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.546880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.547884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.548972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.549579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.549748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.549893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.external = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.550954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.password = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] auth.token = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.551901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.552940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.external = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.553943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.password = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.token = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.554932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.555791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.556274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.558970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.559963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.560889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.561999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562437 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.562960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.563895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.564938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.565920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.566938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.567916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.568947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.569996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.570761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.571957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.572998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.573929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.574878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.575857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.576961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.577924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.578955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.579933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.580946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.581857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] role.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] role.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.582975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.583961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.584919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.585875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.586859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.587859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.588955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.589960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.590922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.591920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.592879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.593986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.594971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.595065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.connection = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.595211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.595292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.595465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.595549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.595716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.596328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.596467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.596548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.596709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.596920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.597955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.598912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.599943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81801) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 19:30:11.600072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.600310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.600550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.600788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.601029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.601275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.601512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.601757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.601994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.602243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.602481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.602727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.602958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.603201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.603452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 19:30:11.603627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 19:30:11.629531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.629759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.629981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.632406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.632641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.632868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.638300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.641114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.666643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.666870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.667378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.667636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.722109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.722366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.770852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.771083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.771284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.771543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.928658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.928880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.929959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.930177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.930393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.953931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.954146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.954242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.954360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.954442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.954574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.954655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.966501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.966951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.979397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.979620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.979824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.979950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81801) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.980917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.981208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 19:30:11.998052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.998259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 19:30:11.998602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.998689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.998801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 19:30:11.998902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 19:30:11.999072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.999217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.999331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.999415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 19:30:11.999612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.999709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.999868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:11.999951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.000232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.000477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.010286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.010564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.010696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.010830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.010977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.011109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.011250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.011564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.018183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.018501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.021078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.021363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.021458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.021645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.021731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.021928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.022830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.023911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.024097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.046157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.046584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.047628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.047911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.048079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.048366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.053322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.053605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.053806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.053890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.054096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.054472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.060556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.060853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.060947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.061223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.067125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.067420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.067558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.067752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.067842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.068906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.069078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.069162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.069427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.084549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.084829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.085140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.085421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.088809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.089499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.092053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.092352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.092756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.093055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.097077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.097852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.100274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.100971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.103494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.103796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.104082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.104364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.104450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.104637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.104731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.104933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.105017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.105221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.105500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.105795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.121194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.121483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.121751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.122014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.122386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.122689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.122964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.123230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.129137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.130420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.130678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.132003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.132226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.132499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.133517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.133797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.135303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.135629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.135904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.136186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.136470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.136565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.136765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.137051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.137337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.137721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.147607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.147895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.148173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.148449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.148823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.148936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.149105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.149224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.149381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.149508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.149791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.150173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.150462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.150746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.165214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.166601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.168122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.169522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.170979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.172408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.172658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.173853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.174736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.175018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.175408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.176148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.176433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.176835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.182060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.183411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.183546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.184893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.186352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.186749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.187034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.187314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.187779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.188177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.188465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.188748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.195227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.195543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.195827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.196716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.197010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.197292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.202649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81801) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.202936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.203217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.203518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.203792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 19:30:12.204334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.204998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.205218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.205346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.205512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.205605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.205779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.205871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.206041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.206123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.206307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81801) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.206391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.206616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.206887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.207156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.207441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.207721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 19:30:12.241200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5628efce89c0 pid: 81801 (default app) Sep 24 19:30:12.242768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5628efce89c0 pid: 81802 (default app) Sep 24 19:30:12.244554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cb392-5cff-484f-af40-6d794f04efa5 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:12.244835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cb392-5cff-484f-af40-6d794f04efa5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:12.245102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cb392-5cff-484f-af40-6d794f04efa5 None None] PATH_INFO: `/v3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:12.249270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Sat Sep 24 19:30:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:19.021617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:19.022154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:19.022154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:19.026391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:19.031584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:19.032145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:19.032145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:19.105765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: INFO dbcounter [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] Registered counter for database keystone Sep 24 19:30:19.125432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG oslo_db.sqlalchemy.engines [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81801) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 19:30:19.139128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writer thread running {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 19:30:19.319400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG passlib.handlers.bcrypt [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81801) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 19:30:19.319737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG passlib.handlers.bcrypt [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81801) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 19:30:19.377605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:19.379219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:19.509170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 19:30:19.956246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:20.024900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: INFO dbcounter [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] Registered counter for database keystone Sep 24 19:30:20.046528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG oslo_db.sqlalchemy.engines [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81802) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 19:30:20.052877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writer thread running {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 19:30:20.173197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586761-c3cf-4716-acf3-9c141b119983 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:20.177553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:20.178177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:20.178454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:20.178727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:20.187184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:20.238540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 19:30:20.238829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfae0e-5dd7-4c08-95bb-4f8c87eaf3d6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:20.248286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Sat Sep 24 19:30:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:30:20.254439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:20.271688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbd114-21cd-424b-9dfc-de3bf4fb8bc3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:20.276120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:20.277251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:20.277526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:20.277790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] PATH_INFO: `/v3/projects/6d64e2847698419baef7cbb0bac87311` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:20.279400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] RBAC: Authorizing `identity:get_project(project_id=6d64e2847698419baef7cbb0bac87311)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:20.332913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 19:30:20.333254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb690c4-1ba3-400f-9a1c-742bc3c939b9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:20.337750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Sat Sep 24 19:30:20 2022] GET /identity/v3/projects/6d64e2847698419baef7cbb0bac87311 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:22.454509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cba37-175c-48ad-a78f-a7337d520eb2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:22.455036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cba37-175c-48ad-a78f-a7337d520eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:22.455128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cba37-175c-48ad-a78f-a7337d520eb2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:22.459160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:22.464689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:22.465105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:22.465243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:22.485315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:22.487341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:22.495303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 19:30:22.763701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cba37-175c-48ad-a78f-a7337d520eb2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:22.777803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cba37-175c-48ad-a78f-a7337d520eb2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:22.782014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c412b-6301-4fbb-968f-12e21818c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:22.783154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c412b-6301-4fbb-968f-12e21818c4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:22.783577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c412b-6301-4fbb-968f-12e21818c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:22.783856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c412b-6301-4fbb-968f-12e21818c4e9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:22.791715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c412b-6301-4fbb-968f-12e21818c4e9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:22.793509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c412b-6301-4fbb-968f-12e21818c4e9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:22.802516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Sat Sep 24 19:30:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 19:30:22.808321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:22.817849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf46013-e9c7-41a1-85f4-547cd3997db9 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:22.822161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa066436-85fe-4314-a6ae-323960b348ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:22.822783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa066436-85fe-4314-a6ae-323960b348ea admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:22.823062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa066436-85fe-4314-a6ae-323960b348ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:22.823330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa066436-85fe-4314-a6ae-323960b348ea admin admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:22.824670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa066436-85fe-4314-a6ae-323960b348ea admin admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:22.826476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa066436-85fe-4314-a6ae-323960b348ea admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:22.828013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Sat Sep 24 19:30:22 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:25.047600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c6b3e-467a-4d9a-bb9a-8a549fdcdc57 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.048154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c6b3e-467a-4d9a-bb9a-8a549fdcdc57 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.048794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c6b3e-467a-4d9a-bb9a-8a549fdcdc57 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.050502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:25.055808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.056476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.056618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.066125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff331b-edd1-4da5-ab5c-5a3c22860403 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.066414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff331b-edd1-4da5-ab5c-5a3c22860403 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.066688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff331b-edd1-4da5-ab5c-5a3c22860403 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.068092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:25.073275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.073563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.073837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.075154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:25.076373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:25.117190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG passlib.handlers.bcrypt [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81802) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 19:30:25.117584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG passlib.handlers.bcrypt [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81802) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 19:30:25.158271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:25.166753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:25.168397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:25.270461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:25.427354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:25.452362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:25.534566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd520da-e824-4b4f-a2d1-7b9f66cee575 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:25.542293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:25.572599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:25.573231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.573516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.573790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.584880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:25.593737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:25.594653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:25.595988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:25.601145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:25.642450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d185c-0185-48e4-80e7-66e3341896db None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:25.644802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:25.681707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:25.682491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.682768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.683036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.683478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:25.702253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61060ff-570d-4f7a-b83f-a645c882cc80 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:25.736392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:25.737007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.737286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.737561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.738224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:25.738525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:25.739944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:25.751276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:25.751400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:25.753092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:25.757104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:25.852161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 8/16] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:30:25.856561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ace6c-4051-4361-9696-4b3b3e49ef12 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:25.857817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:25.860488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:25.892074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:25.892984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.893263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.893533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:25.894668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:25.896147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:25.897746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 9/17] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:25 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:25.903042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:25.917598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:25.960634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d65277-b5a0-44a2-99d2-e54cb3eada6c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:25.994945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:25.995845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:25.996123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:25.996391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:26.002929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92b774-d80a-4b41-bef5-d59be40486f9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:26.014700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:26.016196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:30:26.036686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:26.037722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:30:26.038521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:30:26.038521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:30:26.038521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:26.038521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 9/18] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:30:26.040862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:26.041498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:26.041774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:26.042042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:26.043155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:26.043993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:26.044632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:26.046152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 10/19] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:26.051804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:26.104592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:26.136022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:26.150211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddfbb5-5509-4375-bb07-da1b009038c0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:26.161049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1e29-77da-43aa-915b-3079ee58859d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:26.183992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:26.184592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:26.184869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:26.185142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:26.185780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:26.186078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:26.187597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96be23a-1df2-48cb-8085-8f488e5c4362 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:26.197879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:26.198489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:26.198782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:26.199086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:26.200143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.domain.id=default, user.id=818c3ae70a684477af4280e7bbdcea2d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:26.200454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:26.201875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabe215-aa4a-4415-a005-76b66a158930 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:26.205362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 10/20] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:26.215241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 11/21] 10.222.0.55 () {56 vars in 1322 bytes} [Sat Sep 24 19:30:26 2022] GET /identity/v3/role_assignments?user.id=818c3ae70a684477af4280e7bbdcea2d&scope.domain.id=default&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:28.877237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d688-be47-4331-b5ca-7c247860a882 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:28.877762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d688-be47-4331-b5ca-7c247860a882 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:28.877762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0d688-be47-4331-b5ca-7c247860a882 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:28.879756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 11/22] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:28.884921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80b21-0b6c-4ce9-a079-28e5feecbb9d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:28.885822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80b21-0b6c-4ce9-a079-28e5feecbb9d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:28.886284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80b21-0b6c-4ce9-a079-28e5feecbb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:28.899522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8838957-2257-491b-9ecc-86ae3b2e9821 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:28.899522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8838957-2257-491b-9ecc-86ae3b2e9821 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:28.899522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8838957-2257-491b-9ecc-86ae3b2e9821 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:28.901187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 12/23] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:28.906846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372989a-6888-44eb-b708-bdb8b3a52a23 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:28.908134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372989a-6888-44eb-b708-bdb8b3a52a23 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:28.908610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372989a-6888-44eb-b708-bdb8b3a52a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:28.920604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a5e80b21-0b6c-4ce9-a079-28e5feecbb9d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:28.922043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a5e80b21-0b6c-4ce9-a079-28e5feecbb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:28.947464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a372989a-6888-44eb-b708-bdb8b3a52a23 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:28.947464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a372989a-6888-44eb-b708-bdb8b3a52a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:29.092860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 12/24] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:29.144868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 13/25] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:29.172460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9733e-affa-4ee7-8aa6-76a796ce8eb3 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:29.173582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9733e-affa-4ee7-8aa6-76a796ce8eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:29.174046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9733e-affa-4ee7-8aa6-76a796ce8eb3 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:29.175588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 13/26] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:29.181222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:29.181513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:29.182574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:29.221293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:29.222566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:29.286151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:29.335622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 14/27] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:29.553583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9733e-affa-4ee7-8aa6-76a796ce8eb3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:29.569241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:29.679308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:29.713201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9733e-affa-4ee7-8aa6-76a796ce8eb3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:29.802939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906dd3f-a922-4eca-af39-e037e830fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:29.804350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906dd3f-a922-4eca-af39-e037e830fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:29.804827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906dd3f-a922-4eca-af39-e037e830fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:29.805306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906dd3f-a922-4eca-af39-e037e830fbb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:29.827907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b21d6f-d0a3-48c9-893c-05db2044f8ff None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:29.830779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906dd3f-a922-4eca-af39-e037e830fbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-6906dd3f-a922-4eca-af39-e037e830fbb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:29.835776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:29.836599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:29.837346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 14/28] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:29.838055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7925d6-92d5-438f-b721-0916838b0e9e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:29.846570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7925d6-92d5-438f-b721-0916838b0e9e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:29.846847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7925d6-92d5-438f-b721-0916838b0e9e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:29.856642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 15/29] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:29 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:29.856642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7925d6-92d5-438f-b721-0916838b0e9e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:29.902185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:29.902827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:29.903267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:29.904738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:29.919256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:29.925851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:30:29.957340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:29.958432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:30:29.959331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:30:29.959331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:30:29.959331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:29.964634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 15/30] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:30:29.972976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:30.013731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:30.087020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7925d6-92d5-438f-b721-0916838b0e9e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:30.113036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e8a4f-192f-4a23-8e3e-c2c64d2393de None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:30.160984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:30.162094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.163242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:30.163843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.164124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.164815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:30.165040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.165565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.166027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.166234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:30.167099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:30.168629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:30.171347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d1f03-f5bb-4be4-91a4-05d4b5569bcc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:30.174738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:30.197645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 16/31] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:30.199993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.201519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.202014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.225123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:30.226757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:30.323700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 17/32] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:30.324907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:30.380422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:30.419960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 16/33] 10.222.0.55 () {60 vars in 1131 bytes} [Sat Sep 24 19:30:29 2022] POST /identity/v3/roles => generated 219 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:30.421439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:30.571620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ffa47-ab9a-4297-be8f-4f91b53f96e4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:30.641439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:30.642338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.642758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.643172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.644016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:30.644473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:30.647350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dea73b-25e8-48aa-b505-ec2c3cc1d3d3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:30.667496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 18/34] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:30.667496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43329727-2664-4ca3-a0c7-f7b98e19bc61 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.667496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43329727-2664-4ca3-a0c7-f7b98e19bc61 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.667496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43329727-2664-4ca3-a0c7-f7b98e19bc61 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.667903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14575b91-d50f-4959-8599-858d2e6374a3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:30.667999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 19/35] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:30.670289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f07ed-fd4c-4c6d-954b-d8b8274548b4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.670289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f07ed-fd4c-4c6d-954b-d8b8274548b4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.670289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f07ed-fd4c-4c6d-954b-d8b8274548b4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.670289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 20/36] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:30.678649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f07ed-fd4c-4c6d-954b-d8b8274548b4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:30.737868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc910b-dfb6-4806-a2ed-8595ff56bea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:30.738623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc910b-dfb6-4806-a2ed-8595ff56bea9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.738975 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc910b-dfb6-4806-a2ed-8595ff56bea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.739319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc910b-dfb6-4806-a2ed-8595ff56bea9 None admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.740708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc910b-dfb6-4806-a2ed-8595ff56bea9 None admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:30.742287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc910b-dfb6-4806-a2ed-8595ff56bea9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:30.745378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 17/37] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:30 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:30.746380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579a528-0a4a-4977-a143-45305d4caf92 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.746670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579a528-0a4a-4977-a143-45305d4caf92 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.746950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579a528-0a4a-4977-a143-45305d4caf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.765526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7579a528-0a4a-4977-a143-45305d4caf92 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:30.768771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7579a528-0a4a-4977-a143-45305d4caf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:30.794891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:30.817515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f07ed-fd4c-4c6d-954b-d8b8274548b4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:30.832215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:30.845316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 18/38] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:30.846800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.847148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.847842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.857819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:30.858683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:30.859661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:30.860112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:30.862061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:30.863823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:30.865694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:30.866980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:30.944838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 19/39] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:30.946456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:31.034111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 21/40] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:30:30 2022] POST /identity/v3/projects => generated 301 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:31.036634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:31.194508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c41b77-beb0-4029-a2dc-67af841176b7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:31.209782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495520a-2042-4c22-b63c-41ca95417ce6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:31.236371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:31.256435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:31.256930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:31.257225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:31.257509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:31.258357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:31.260252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:31.261495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:31.271407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:31.274261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:31.274857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:31.275144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:31.276946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:31.280439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:31.282185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 22/41] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:31.282768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:31.283949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 20/42] 10.222.0.55 () {60 vars in 1131 bytes} [Sat Sep 24 19:30:30 2022] POST /identity/v3/roles => generated 225 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:31.287923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:31.442433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c2432-08ef-4253-9a1e-d05c929d8837 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:31.449346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b740-f2db-4f52-a8ff-2f5175771f1e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:31.490410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:31.490987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:31.491274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:31.491591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:31.492772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:31.494252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:31.495937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 21/43] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:31.496673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:31.502245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:31.503161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:31.503666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:31.503964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:31.504653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:31.506049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:31.531419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 23/44] 10.222.0.55 () {60 vars in 1131 bytes} [Sat Sep 24 19:30:31 2022] POST /identity/v3/roles => generated 223 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:31.532257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:31.591935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:31.621278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:31.678830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079a5a2-4cb7-47a1-83e1-fac0cfa9b380 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:31.723815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba9c7-d3ff-43fd-b004-6051ca911ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:31.724460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba9c7-d3ff-43fd-b004-6051ca911ae6 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:31.724744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba9c7-d3ff-43fd-b004-6051ca911ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:31.725022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba9c7-d3ff-43fd-b004-6051ca911ae6 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:31.725769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba9c7-d3ff-43fd-b004-6051ca911ae6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:31.727299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba9c7-d3ff-43fd-b004-6051ca911ae6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:31.762033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 24/45] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:30:31 2022] POST /identity/v3/projects => generated 312 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:31.786907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0a45-d985-4b19-88f5-3faf4a0d3f2d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:31.835062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e098ff-c0ea-4faa-91aa-4531c1f6ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:31.836101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e098ff-c0ea-4faa-91aa-4531c1f6ce9d None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:31.836568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e098ff-c0ea-4faa-91aa-4531c1f6ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:31.837026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e098ff-c0ea-4faa-91aa-4531c1f6ce9d None admin] PATH_INFO: `/v3/domains/default/users/818c3ae70a684477af4280e7bbdcea2d/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:31.839581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e098ff-c0ea-4faa-91aa-4531c1f6ce9d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=818c3ae70a684477af4280e7bbdcea2d, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:31.841091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e098ff-c0ea-4faa-91aa-4531c1f6ce9d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:31.858231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 22/46] 10.222.0.55 () {58 vars in 1361 bytes} [Sat Sep 24 19:30:31 2022] PUT /identity/v3/domains/default/users/818c3ae70a684477af4280e7bbdcea2d/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:30:33.996103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b200e7-e9c1-4921-a9b1-84acb59e36ce None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:33.996633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b200e7-e9c1-4921-a9b1-84acb59e36ce None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:33.996633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b200e7-e9c1-4921-a9b1-84acb59e36ce None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:33.998004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 25/47] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:34.003861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.004287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.004444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.022914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:34.024347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:34.054687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10bee-27b2-4295-a572-1c74a4daea29 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.054977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10bee-27b2-4295-a572-1c74a4daea29 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.055260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10bee-27b2-4295-a572-1c74a4daea29 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.057058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 26/48] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:34.062694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.062971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.063323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.082597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:34.084144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:34.111769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 23/49] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:34.158255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 27/50] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:34.396419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:34.428044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:34.444347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:34.475135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:34.509913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1886dd4-1c49-4a55-b69a-2777ea15d891 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:34.542292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8788c2f-321c-4df4-b314-69f1582420b1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:34.552198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:34.553258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.553737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.554204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.555589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:34.557547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:34.559133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 24/51] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:34.565902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:34.583056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:34.583743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.584023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.584294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.596491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:34.602954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:34.603994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:34.604740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 28/52] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:30:34.610049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:34.682880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944e1b6-876d-41a8-8448-0308945ae1c3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:34.692666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:34.723063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:34.723862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.724225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.724593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.725375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:34.727015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:34.762163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 25/53] 10.222.0.55 () {60 vars in 1140 bytes} [Sat Sep 24 19:30:34 2022] POST /identity/v3/projects => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:30:34.839418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdb8aa-99f4-45e8-9a17-12b4d423b7ca None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:34.841186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:34.884682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:34.885430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:34.885788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:34.886134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:34.886903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:34.887277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:34.888894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:34.899773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 29/54] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:34.905641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:35.022211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0623f9-46d6-4d2e-b91c-0226997f74f8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:35.057943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:35.059069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:35.059365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:35.059677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:35.060872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:35.062356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:35.064584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 26/55] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:34 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:35.072722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:35.183546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b89a0-3a04-46cb-9b6b-103ee76329cb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:35.221607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621fe6b-6a5d-4f6e-bbf1-4ab004f86d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:35.222250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621fe6b-6a5d-4f6e-bbf1-4ab004f86d0c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:35.222535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621fe6b-6a5d-4f6e-bbf1-4ab004f86d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:35.222807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621fe6b-6a5d-4f6e-bbf1-4ab004f86d0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:35.224003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621fe6b-6a5d-4f6e-bbf1-4ab004f86d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:35.225452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621fe6b-6a5d-4f6e-bbf1-4ab004f86d0c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:35.227043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 30/56] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:35.233410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:35.267556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:35.351873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1eb39-5401-4ceb-97c8-dd451bee6ba7 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:35.387948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:35.388852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:35.389133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:35.389405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:35.390125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.domain.id=default, user.id=818c3ae70a684477af4280e7bbdcea2d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:35.390430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:35.392383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509e26-fcb5-494e-8a07-5a5c30ea602d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:35.398552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 27/57] 10.222.0.55 () {56 vars in 1322 bytes} [Sat Sep 24 19:30:35 2022] GET /identity/v3/role_assignments?user.id=818c3ae70a684477af4280e7bbdcea2d&scope.domain.id=default&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 530 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:36.985435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928048e-c5a6-404f-b316-87619aacc391 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:36.985979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928048e-c5a6-404f-b316-87619aacc391 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:36.986062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928048e-c5a6-404f-b316-87619aacc391 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:36.987416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 31/58] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:36.992635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:36.993066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:36.993235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:37.013065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:37.014272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:37.082522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 28/59] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:37.351379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928048e-c5a6-404f-b316-87619aacc391 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:37.383606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:37.454879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928048e-c5a6-404f-b316-87619aacc391 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:37.488666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5031d2a-1ba9-40fe-960e-a47c36ecc986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:37.489300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031d2a-1ba9-40fe-960e-a47c36ecc986 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:37.489582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031d2a-1ba9-40fe-960e-a47c36ecc986 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:37.489859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031d2a-1ba9-40fe-960e-a47c36ecc986 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:37.491005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5031d2a-1ba9-40fe-960e-a47c36ecc986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:37.492505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5031d2a-1ba9-40fe-960e-a47c36ecc986 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:37.494078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 32/60] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:37.499609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:37.523379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:37.603848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7b5b8-771d-409a-bdda-123e03770a80 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:37.638226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb86585-a70a-4c53-bcc0-b9933f01948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:37.638836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86585-a70a-4c53-bcc0-b9933f01948f None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:37.639119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86585-a70a-4c53-bcc0-b9933f01948f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:37.639390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86585-a70a-4c53-bcc0-b9933f01948f None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:37.640066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb86585-a70a-4c53-bcc0-b9933f01948f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:37.641566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb86585-a70a-4c53-bcc0-b9933f01948f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:37.668549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 29/61] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:30:37 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:41.249280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12c20-04ac-477a-bcbc-12fd21fcaa82 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.250323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12c20-04ac-477a-bcbc-12fd21fcaa82 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.251083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12c20-04ac-477a-bcbc-12fd21fcaa82 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.252681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 33/62] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:41.267914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703bfd8-a035-4d6d-834b-fa3beab16814 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.268839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703bfd8-a035-4d6d-834b-fa3beab16814 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.269331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703bfd8-a035-4d6d-834b-fa3beab16814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.279415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746b9d3-c860-44c2-8323-a3026b4d0f73 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.279415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746b9d3-c860-44c2-8323-a3026b4d0f73 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.279415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746b9d3-c860-44c2-8323-a3026b4d0f73 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.280040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 34/63] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:41.285372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8402-01b2-4d8d-91e0-0ce8e74d839a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.285901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8402-01b2-4d8d-91e0-0ce8e74d839a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.286381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8402-01b2-4d8d-91e0-0ce8e74d839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.306445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2703bfd8-a035-4d6d-834b-fa3beab16814 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:41.308011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2703bfd8-a035-4d6d-834b-fa3beab16814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:41.320996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3c1f8402-01b2-4d8d-91e0-0ce8e74d839a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:41.322485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3c1f8402-01b2-4d8d-91e0-0ce8e74d839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:41.409204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 30/64] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:41.410795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 35/65] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:41.410982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d892e-a4dd-49ea-bb31-eb3b0805422e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.419725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d892e-a4dd-49ea-bb31-eb3b0805422e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.420201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d892e-a4dd-49ea-bb31-eb3b0805422e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.421953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 31/66] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:41.427263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.427658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.427935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.464958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:41.466163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:41.491621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:41.540296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f489f-074d-4519-b26d-7eaf3d228730 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.540581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f489f-074d-4519-b26d-7eaf3d228730 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.541020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f489f-074d-4519-b26d-7eaf3d228730 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.542533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 32/67] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:41.548103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b7fc3-534b-45ec-b347-fc9053120e6d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.548810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b7fc3-534b-45ec-b347-fc9053120e6d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.549234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b7fc3-534b-45ec-b347-fc9053120e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.560970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 36/68] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:41.578558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bf8b7fc3-534b-45ec-b347-fc9053120e6d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:41.581413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bf8b7fc3-534b-45ec-b347-fc9053120e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:41.585441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:41.684364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 33/69] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:41.709497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:41.751472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273395cf-14df-40f0-9d20-2e406f67ba42 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.751835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273395cf-14df-40f0-9d20-2e406f67ba42 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.752025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273395cf-14df-40f0-9d20-2e406f67ba42 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.753373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 34/70] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:41.758448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.758732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.759004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.787059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:41.788283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:41.815001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9c37e-0e44-4239-893a-af08e1dfa285 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:41.849590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:41.850140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:41.850418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:41.850687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:41.861257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 35/71] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:41.862153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:41.862645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:41.868646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:41.869491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:41.870228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:41.870810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 37/72] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:41.875965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:41.902886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:41.954039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.003634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044296d6-ef92-4796-81e4-2e6223c0b752 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.004968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d93f55-d386-4158-bf96-cb8121389be8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.041791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.042395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.042674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.042965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.043628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:42.043936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.044094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.044722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.044999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.045279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.045370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.055853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 38/73] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:30:42.055853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.057101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:42.064969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:42.065807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:42.066641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 36/74] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:42.071605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.150198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.169314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bae6a3-b6a1-49ed-bb6a-553a3257a3c4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.175490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30716aa-4356-4452-b24f-e97f31e982ae None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.203980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.211866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.212057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.212499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.212690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.212773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.212971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.213063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.213242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.223543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:42.224114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:42.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:42.229656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:42.230393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:42.230946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:42.232173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:42.232915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:42.233481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 39/75] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:42.233481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.234370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 37/76] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:42.236065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.332280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aff975-d567-495c-9b39-7cf972db1216 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.333198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43a559-2f57-4c4a-90d0-63e945846c22 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.366117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.366742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.367023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.367299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.367624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.368227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.368506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.368692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.368778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.369417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:42.369719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.370155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.371090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.371782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 40/77] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:42.372537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.385132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.392281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 38/78] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:42.393134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.445859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.475271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac92dc1-8e5b-4529-affc-e72e7cf9204c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.496214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e027d-2b37-437b-94be-52602f721191 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.508322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.508952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.509232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.509501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.510613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.512096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.513662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 41/79] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:42.514415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.533147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.533760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.534036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.534306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.534969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:42.535269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.536704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.549219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 39/80] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:42.549219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.603041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.624766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba538e-8340-4458-95fd-eb9785780388 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.652774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a2b9-39a6-4bbd-85a8-c89e4d60114e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.658746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.659353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.659659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.659930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.660591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:42.660891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.662264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.666182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.670460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 42/81] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:42.671272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.689416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.690039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.690322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.690594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.759474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.760902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.769764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654ba2f-3fbc-4afc-a220-af5cf79f6ddb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.803304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.804038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.804324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.804600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.816015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.821104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.822776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.824449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 43/82] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:42.825204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.868959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 40/83] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:42.869809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:42.883298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:42.924860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c77b0-1021-43d8-a684-faefa6cfc5a0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.964897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:42.965800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:42.966076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:42.966344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:42.966994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:42.968460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:42.972326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79940d0-9e3a-4a48-b606-35ac41451406 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:42.996406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 44/84] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:30:42 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:42.997234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.029849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.030436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.030722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.030991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.032753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.034261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.040734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 41/85] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:43.040734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.068230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.125466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea0074-7ad8-4ecb-a9df-e4bbad8c01e9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.134443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.155542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178997-a363-4970-99d3-2e575659f40c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.173595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.174249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.174726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.175014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.176523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.177994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.180108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 45/86] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:42 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:43.181201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.195719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.196335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.196678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.196877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.281204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.283020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.311782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e217-38c8-4766-90b2-6ab9ebe6fc27 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.341068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.354999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.355667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.356193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.356193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.357403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.358900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.361234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 46/87] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:43.361913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.384327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.410509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 42/88] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:43.410509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.486640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc044-6436-466d-88ab-d0011e46a1be None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.522439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e4ddf-36d8-439e-98db-a7d70aa304b4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.529059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.529756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.530109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.530915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.567274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.567841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.568127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.568405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.604375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.620138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.621725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.624052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.646156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.647786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.741012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 47/89] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:43.741912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.763809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 43/90] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:43.764666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.853482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.868749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:43.876609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef759f38-281a-4d96-976f-49baff781bb3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.882093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23353ec1-39ef-490e-b761-8c2b981af21f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:43.918698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.919313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.919628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.919891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.920781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=818c3ae70a684477af4280e7bbdcea2d) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:43.921095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.921273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:43.921891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:43.922770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:43.923046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:43.923228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.924413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:43.926175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:43.928149 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 44/91] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:43.929658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:43.932482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 48/92] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/role_assignments?user.id=818c3ae70a684477af4280e7bbdcea2d&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:43.933293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:44.063457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a546007-9bc4-408e-92ec-249fc0fa9820 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:44.104391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:44.104958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:44.105396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:44.105805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:44.107106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:44.108747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:44.112528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 45/93] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:44.113495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:44.137483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:44.186961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e07ce8-b978-4ec0-ac30-605ce34f3619 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:44.242347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:44.243245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:44.243696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:44.244169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] PATH_INFO: `/v3/projects/586e6f4a4108441a8b41cedf180e3540` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:44.246522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] RBAC: Authorizing `identity:get_project(project_id=586e6f4a4108441a8b41cedf180e3540)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:44.246637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3262547-e47d-42e0-aa78-0dbd40f59171 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:44.248241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:44.250048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 49/94] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:43 2022] GET /identity/v3/projects/586e6f4a4108441a8b41cedf180e3540 => generated 312 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:44.251652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:44.280006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:44.289265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:44.290082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:44.290488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:44.290910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:44.291935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=475b2f85e68d43ac91521a15a826e5e3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:44.292385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:44.294123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:44.301294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 46/95] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:44 2022] GET /identity/v3/role_assignments?user.id=475b2f85e68d43ac91521a15a826e5e3&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:44.302253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:44.401511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:44.428295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be697962-bf7b-4c84-a8b2-7758cb7326e0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:44.456509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749126-7eb7-4d0c-b81f-ebd374aedf1d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:44.481210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:44.482121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:44.484130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:44.484701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:44.485703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=586e6f4a4108441a8b41cedf180e3540, user.id=475b2f85e68d43ac91521a15a826e5e3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:44.487359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:44.489061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80836a63-6820-4971-b272-6899a42feed3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:44.495183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 47/96] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:44 2022] GET /identity/v3/role_assignments?user.id=475b2f85e68d43ac91521a15a826e5e3&scope.project.id=586e6f4a4108441a8b41cedf180e3540&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:44.517610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:44.518481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:44.518900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:44.519308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:44.520247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] RBAC: Adding query filter params (role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=475b2f85e68d43ac91521a15a826e5e3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:44.520703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:44.522327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb96a6-4302-43d3-8433-b47bc3dfb79f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:44.528931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 50/97] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:44 2022] GET /identity/v3/role_assignments?user.id=475b2f85e68d43ac91521a15a826e5e3&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:46.032453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269f0c7-37f1-43c3-b227-a16a1e6c72ad None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:46.033503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269f0c7-37f1-43c3-b227-a16a1e6c72ad None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:46.033984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269f0c7-37f1-43c3-b227-a16a1e6c72ad None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:46.035603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 48/98] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:46.048269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f792-5011-4562-9f58-1c1b6071cf17 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:46.049175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f792-5011-4562-9f58-1c1b6071cf17 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:46.049680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f792-5011-4562-9f58-1c1b6071cf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:46.084494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-54f4f792-5011-4562-9f58-1c1b6071cf17 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:46.086031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-54f4f792-5011-4562-9f58-1c1b6071cf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:46.177116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 51/99] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:46.178779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6cf1-808a-489d-91bb-ed78ef86ea5e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:46.179212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6cf1-808a-489d-91bb-ed78ef86ea5e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:46.179645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6cf1-808a-489d-91bb-ed78ef86ea5e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:46.181421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 49/100] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:46.186541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:46.186836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:46.187120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:46.216107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:46.217626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:46.234413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:46.315223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 52/101] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:46.585053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6cf1-808a-489d-91bb-ed78ef86ea5e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:46.716182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6cf1-808a-489d-91bb-ed78ef86ea5e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:46.762365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:46.763013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:46.763294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:46.763647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:46.766805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:46.768316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:46.769920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 50/102] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:46.775551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:46.886381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:46.911573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:46.937620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375a7d8-adf8-4da3-91b5-7a6bd0679a79 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:47.004331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:47.004954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.005423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.006017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.006817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:47.008792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:47.051901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:47.067018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 53/103] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:30:46 2022] POST /identity/v3/projects => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:47.118841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751ded9-6961-4811-8642-b2ecffb2a0fa None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:47.177419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5b4db-864b-48d2-82f3-776adeff9ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:47.178100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b4db-864b-48d2-82f3-776adeff9ddf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.178383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b4db-864b-48d2-82f3-776adeff9ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.178657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b4db-864b-48d2-82f3-776adeff9ddf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.196650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5b4db-864b-48d2-82f3-776adeff9ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-42d5b4db-864b-48d2-82f3-776adeff9ddf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:47.202618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:47.204128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:47.204872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 51/104] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:47.210971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:47.300471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac7a6-4be5-4f3e-ac11-126c66a4b125 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.300794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac7a6-4be5-4f3e-ac11-126c66a4b125 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.301106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac7a6-4be5-4f3e-ac11-126c66a4b125 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.302506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 52/105] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:30:47.308659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71696fdb-af0d-4946-a056-bbe1a969630a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.308952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71696fdb-af0d-4946-a056-bbe1a969630a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.309231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71696fdb-af0d-4946-a056-bbe1a969630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.341723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-71696fdb-af0d-4946-a056-bbe1a969630a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:47.342265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-71696fdb-af0d-4946-a056-bbe1a969630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:47.392269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85e5a-58a4-46d7-9cb0-fb21fe828891 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:47.454199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 53/106] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:47.455890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05a98b-f787-4202-b797-1238cdf49bcd None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.456388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05a98b-f787-4202-b797-1238cdf49bcd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.456905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05a98b-f787-4202-b797-1238cdf49bcd None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.458446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 54/107] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:47.468687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a85f9-3157-47c8-a76e-6931a5f23443 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.470099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a85f9-3157-47c8-a76e-6931a5f23443 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.471085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a85f9-3157-47c8-a76e-6931a5f23443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.473158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:47.474027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.475820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.476103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.476818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:47.477110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:47.478489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:47.480464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:47.494294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b68a85f9-3157-47c8-a76e-6931a5f23443 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:47.495507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 54/108] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:47.496333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b68a85f9-3157-47c8-a76e-6931a5f23443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:47.501214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:47.573018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 55/109] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:47.574723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04165501-2938-45d4-8d8f-527333b1fb91 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.575199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04165501-2938-45d4-8d8f-527333b1fb91 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.575723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04165501-2938-45d4-8d8f-527333b1fb91 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.577196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 56/110] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:47.590579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.591140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.591681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.592379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:47.617263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:47.623634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:47.701727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55566051-fa4d-4e60-a9df-4653a713ad6a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:47.713815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 57/111] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:47.777043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:47.777696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:47.777984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:47.778260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] PATH_INFO: `/v3/users/bf1ee5df3590494e83b5a4c817f5d0e5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:47.798104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:47.802711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] RBAC: Authorizing `identity:get_user(user_id=bf1ee5df3590494e83b5a4c817f5d0e5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:47.803474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:47.805381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 55/112] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:47 2022] GET /identity/v3/users/bf1ee5df3590494e83b5a4c817f5d0e5 => generated 289 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:47.811549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:47.919815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:47.955314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee8ab6-2322-493a-a2be-0f6ffa6c430c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.022401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec28b42-f216-4588-941c-9b68392e41bc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.045029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.045684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.045971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.046252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.055880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.080792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:48.086467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:48.087284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:48.088079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 58/113] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:30:48.088787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.095509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.096260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.096543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.096815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.178062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.180005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.213003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a225d-33ef-4fdb-986d-f8186b7755bf None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.241289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.254749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.255746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.256257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.256739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.269049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:48.274050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:48.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:48.275645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 59/114] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:48.276342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.310700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 56/115] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:48.311792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.334051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.388635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cae311-5e70-428e-a0bc-5f51fb4b07a1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.427389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33416ba2-e454-47dd-a832-43c689934e63 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.429675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc57e14-2e47-4263-a6d8-4a5b518cd878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.430317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc57e14-2e47-4263-a6d8-4a5b518cd878 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.430859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc57e14-2e47-4263-a6d8-4a5b518cd878 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.431136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc57e14-2e47-4263-a6d8-4a5b518cd878 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.441728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc57e14-2e47-4263-a6d8-4a5b518cd878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-8fc57e14-2e47-4263-a6d8-4a5b518cd878 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:48.446768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:48.447643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:48.448427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 60/116] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:48.449275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40623cd-e2b8-4c2d-8dad-27945071e226 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.449567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40623cd-e2b8-4c2d-8dad-27945071e226 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.449844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40623cd-e2b8-4c2d-8dad-27945071e226 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.451098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 61/117] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:48.455658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40623cd-e2b8-4c2d-8dad-27945071e226 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.473704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.474318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.474594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.474861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.475560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:48.476280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.477657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.491084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 57/118] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:48.491084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.505288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.568660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40623cd-e2b8-4c2d-8dad-27945071e226 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.577415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.599705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1231fa1-71d6-4cea-b24f-ffab41e3976d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.605541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.606147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.606426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.606694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.607356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:48.607683 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.609084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.622062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 62/119] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:48.622062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.641254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53add41e-7670-460b-8e25-aff1edbe448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.641890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add41e-7670-460b-8e25-aff1edbe448f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.642172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add41e-7670-460b-8e25-aff1edbe448f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.642449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add41e-7670-460b-8e25-aff1edbe448f None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.644069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53add41e-7670-460b-8e25-aff1edbe448f None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.645517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53add41e-7670-460b-8e25-aff1edbe448f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.647171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 58/120] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:48.648380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.648668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.648942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.669317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:48.670901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:48.734184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3755da-8592-4e60-a274-fd5780a4e190 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.737569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.745061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 59/121] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:48.756322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.770904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.771598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.771831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.772180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.772904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:48.773212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.774640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.784288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 63/122] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:48.785195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.864909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:48.893636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9ea6e-49e5-4890-81e1-06f5f297b06f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.894431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac0a96-174d-43da-8465-6e23778e274c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:48.932395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.933417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.933841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.934492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.935691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.937238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.939749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 64/123] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:48.940736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:48.941615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:48.942092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:48.942561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:48.943981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:48.944339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.945973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:48.949593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 60/124] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:48.950592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:48.987157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.067106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389d9c-3a34-497c-a312-24215e8eeeb0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.080234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67d5f-8b8c-43ab-a541-c66a4eeb89ab None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.111090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.111938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.112225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.112504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.113247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] RBAC: Adding query filter params (role.id=4bad7209656f49bfbc3d5d94c8824368, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=bf1ee5df3590494e83b5a4c817f5d0e5) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:49.113618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.115086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.116443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.117362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.117708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.118479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.119826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.120913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 65/125] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/role_assignments?user.id=bf1ee5df3590494e83b5a4c817f5d0e5&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=4bad7209656f49bfbc3d5d94c8824368 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:49.121364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.123060 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 61/126] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:48 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:49.123919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.132631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.145867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.251067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1053d-b6ef-4189-b7d5-6528a4837d22 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.271457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.271457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed2e98-7293-434c-b798-34616ab17bdb None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.301864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.302658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.302945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.303221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.304878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.307374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.308326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 62/127] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:49 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:49.309256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.329118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.330093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.330512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.330919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] PATH_INFO: `/v3/projects/586e6f4a4108441a8b41cedf180e3540` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.332379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] RBAC: Authorizing `identity:get_project(project_id=586e6f4a4108441a8b41cedf180e3540)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.334226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.336250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 66/128] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:49 2022] GET /identity/v3/projects/586e6f4a4108441a8b41cedf180e3540 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:49.337200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.410492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.434291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a2b13-0f1c-408b-a55e-edfe7168465a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.472870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.473514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.473797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.474073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.476174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dba42-ed40-49bb-8a7c-1d70b7acea7b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.489789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:49.495089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:49.496050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:49.496800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 63/129] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:30:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:49.497427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.524250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.525364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.525807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.526436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.528465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.530736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.533247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 67/130] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:49 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:49.538593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.579773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.632179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae52dff-e83c-4165-8e6c-c77c1c4e1998 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.660143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960953-c4d7-4a16-af56-6f46aa149b56 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.671652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.673050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.673481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.673762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/818c3ae70a684477af4280e7bbdcea2d/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.676172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] RBAC: Authorizing `identity:create_grant(project_id=d98f31b12d634fe4849900dfd66e2969, user_id=818c3ae70a684477af4280e7bbdcea2d, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.677789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.695196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 64/131] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:30:49 2022] PUT /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/818c3ae70a684477af4280e7bbdcea2d/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:30:49.696107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.699274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.705124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.706164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.706703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.708502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] PATH_INFO: `/v3/projects/586e6f4a4108441a8b41cedf180e3540/users/475b2f85e68d43ac91521a15a826e5e3/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.711358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] RBAC: Authorizing `identity:create_grant(project_id=586e6f4a4108441a8b41cedf180e3540, user_id=475b2f85e68d43ac91521a15a826e5e3, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.713213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.736084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 68/132] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:30:49 2022] PUT /identity/v3/projects/586e6f4a4108441a8b41cedf180e3540/users/475b2f85e68d43ac91521a15a826e5e3/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:30:49.737325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.874545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a975de-b965-408d-8f93-dc5ad5abed95 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:49.924152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:49.925054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.925341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.925615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.926314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:49.926623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:49.928098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f036af7-dfba-4e81-8e7f-51d624ceb721 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:49.939175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 65/133] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:30:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:49.940677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1be59-54ba-4c5f-b91c-4474f75d9c8a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:49.941630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1be59-54ba-4c5f-b91c-4474f75d9c8a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:49.941915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1be59-54ba-4c5f-b91c-4474f75d9c8a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:49.943399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 66/134] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:49.948220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1be59-54ba-4c5f-b91c-4474f75d9c8a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:49.976831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:49.985017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93732d1-d70f-48e2-bc9a-c9d2d41598de None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:50.030149 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c2a53-4104-4018-bd98-cc80d53d9e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:50.030845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c2a53-4104-4018-bd98-cc80d53d9e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:50.032151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c2a53-4104-4018-bd98-cc80d53d9e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:50.032454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c2a53-4104-4018-bd98-cc80d53d9e20 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/475b2f85e68d43ac91521a15a826e5e3/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:50.034614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c2a53-4104-4018-bd98-cc80d53d9e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=d98f31b12d634fe4849900dfd66e2969, user_id=475b2f85e68d43ac91521a15a826e5e3, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:50.036196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c2a53-4104-4018-bd98-cc80d53d9e20 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:50.052999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 69/135] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:30:49 2022] PUT /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/475b2f85e68d43ac91521a15a826e5e3/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:30:50.054532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:50.055023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:50.055324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:50.059970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:50.089186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:50.091094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:50.118369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1be59-54ba-4c5f-b91c-4474f75d9c8a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:50.163683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:50.165189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:50.165624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:50.166045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:50.168076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:50.170107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:50.172640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 67/136] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:49 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:50.179207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:50.187632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 70/137] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:50.299047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a9de9-a07b-4f5a-aaf0-c50f51e9e44f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:50.335972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:50.338748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:50.339681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:50.339785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:50.339990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:50.341234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:50.342729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:50.348834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 68/138] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:50 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:50.351908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:50.452290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:50.473359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1548-1a53-4f1f-9ea9-b9c938907e74 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:50.512528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:50.513206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:50.513487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:50.513763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/475b2f85e68d43ac91521a15a826e5e3/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:50.516286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] RBAC: Authorizing `identity:create_grant(project_id=d98f31b12d634fe4849900dfd66e2969, user_id=475b2f85e68d43ac91521a15a826e5e3, role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:50.517773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:50.534812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 71/139] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:30:50 2022] PUT /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/475b2f85e68d43ac91521a15a826e5e3/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:30:50.652720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:50.786175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8e40-a895-4983-95c4-95fc8fd7f851 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:50.833324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29869dbc-3a01-43f8-bcb5-5c784cb1fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:50.833978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29869dbc-3a01-43f8-bcb5-5c784cb1fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:50.834264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29869dbc-3a01-43f8-bcb5-5c784cb1fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:50.834542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29869dbc-3a01-43f8-bcb5-5c784cb1fdb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:50.835709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29869dbc-3a01-43f8-bcb5-5c784cb1fdb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:50.837172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29869dbc-3a01-43f8-bcb5-5c784cb1fdb5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:50.839083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 69/140] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:50.844649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:50.970294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd497c8-4742-4a71-8ca2-3dab6bc846bb None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:51.011692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbd8aa-c560-4660-971d-c6762b584538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:51.012317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbd8aa-c560-4660-971d-c6762b584538 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:51.012601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbd8aa-c560-4660-971d-c6762b584538 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:51.012885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbd8aa-c560-4660-971d-c6762b584538 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:51.013555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbd8aa-c560-4660-971d-c6762b584538 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:51.015143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbd8aa-c560-4660-971d-c6762b584538 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:51.043950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:51.047468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 72/141] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:30:50 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:52.080683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fed0c-ef66-4339-ba57-dab2bc1d586b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:52.081188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fed0c-ef66-4339-ba57-dab2bc1d586b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:52.081261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fed0c-ef66-4339-ba57-dab2bc1d586b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:52.082576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 70/142] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:52.088513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:52.088913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:52.089097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:52.110174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:52.111367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:52.196200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 73/143] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:52.747945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fed0c-ef66-4339-ba57-dab2bc1d586b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:52.826661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:52.867057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fed0c-ef66-4339-ba57-dab2bc1d586b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:52.905683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c647-6063-425d-8752-3b6e13c5285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:52.907121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c647-6063-425d-8752-3b6e13c5285f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:52.907668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c647-6063-425d-8752-3b6e13c5285f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:52.907956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c647-6063-425d-8752-3b6e13c5285f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:52.920697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c647-6063-425d-8752-3b6e13c5285f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-03e3c647-6063-425d-8752-3b6e13c5285f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:52.927449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:52.928315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:52.929171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 71/144] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:52.938220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:53.049489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3507a5-613b-4202-84f2-1afc7dbea2e9 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.049897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3507a5-613b-4202-84f2-1afc7dbea2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.050305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3507a5-613b-4202-84f2-1afc7dbea2e9 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.052211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 72/145] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:53.058496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.058893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.059693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.087356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeae2f-5fe5-464d-952f-7d04cf30bff1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:53.100468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:53.101697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:53.134799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:53.154428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:53.155060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.155336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.155631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.156376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:53.156674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:53.158061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:53.174932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 74/146] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:53.181272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:53.216850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 73/147] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:53.329808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf7e02-495d-475f-9dee-ccd63d1ef238 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:53.373070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649d245-cca1-465a-9737-56945eb881e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:53.373707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d245-cca1-465a-9737-56945eb881e7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.373984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d245-cca1-465a-9737-56945eb881e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.374258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d245-cca1-465a-9737-56945eb881e7 None admin] PATH_INFO: `/v3/users/bf1ee5df3590494e83b5a4c817f5d0e5` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.379666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649d245-cca1-465a-9737-56945eb881e7 None admin] RBAC: Authorizing `identity:get_user(user_id=bf1ee5df3590494e83b5a4c817f5d0e5)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:53.381145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649d245-cca1-465a-9737-56945eb881e7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:53.383165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 75/148] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:53 2022] GET /identity/v3/users/bf1ee5df3590494e83b5a4c817f5d0e5 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:53.388534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:53.493253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:53.547549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a0bc-289a-4439-9e65-c55b482b392f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:53.551406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef41d5-f057-4574-bb1a-1c9779c91422 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.551738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef41d5-f057-4574-bb1a-1c9779c91422 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.552014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef41d5-f057-4574-bb1a-1c9779c91422 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.553369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 76/149] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:53.575174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.575491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.575771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.595870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:53.596827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.597305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.597784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.601613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:53.601613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:53.602771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:53.604480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:53.606358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 74/150] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:53 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:53.612262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:53.669534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:53.679096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 77/151] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:30:53.798695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab488e24-7abd-4991-ad2d-41fa08c3b4a2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:53.837676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:53.847349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:53.848479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:53.848753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:53.849018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/bf1ee5df3590494e83b5a4c817f5d0e5/roles/4bad7209656f49bfbc3d5d94c8824368` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:53.856823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] RBAC: Authorizing `identity:create_grant(project_id=d98f31b12d634fe4849900dfd66e2969, user_id=bf1ee5df3590494e83b5a4c817f5d0e5, role_id=4bad7209656f49bfbc3d5d94c8824368)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:53.858286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:53.879121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 75/152] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:30:53 2022] PUT /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969/users/bf1ee5df3590494e83b5a4c817f5d0e5/roles/4bad7209656f49bfbc3d5d94c8824368 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:30:53.894630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.045479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28517c-1926-45ad-a13f-17f3b42d600a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:54.092602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:54.093258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.094089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.094588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.106492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:54.111916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:54.112799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:54.113969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 78/153] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:54.120632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.154596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.273099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:54.283120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810e7b9-c002-45a9-ad8a-4bfbdfff111b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:54.326510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:54.328792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.329318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.329602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.330341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:54.330654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:54.332667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bad12-070d-409d-8efd-ea654c0925fe None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:54.342850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 76/154] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:54.344709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d752cf4-5dd4-4237-b479-7eab74172210 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.345274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d752cf4-5dd4-4237-b479-7eab74172210 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.345567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d752cf4-5dd4-4237-b479-7eab74172210 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.347180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 77/155] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:54.352580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f06c28-5c1a-42d2-b83e-d388f0b44e19 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.352871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f06c28-5c1a-42d2-b83e-d388f0b44e19 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.353431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f06c28-5c1a-42d2-b83e-d388f0b44e19 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.355222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 78/156] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:54.356318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677db7c-8a30-4060-9ddf-c8abb3385f49 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.356654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677db7c-8a30-4060-9ddf-c8abb3385f49 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.356871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677db7c-8a30-4060-9ddf-c8abb3385f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.378868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2677db7c-8a30-4060-9ddf-c8abb3385f49 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:54.380405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2677db7c-8a30-4060-9ddf-c8abb3385f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:54.389438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e701a-cacb-4f6b-8c90-c71a04134363 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:54.441020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:54.474475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:54.475112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.475386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.475680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.485700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 79/157] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:54.486753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.487032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.487340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.497439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:54.502388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:54.503463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:54.504309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:54.505643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 79/158] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:54.506915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.525752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:54.526931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:54.662144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 80/159] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:54.663018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.682381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17ff5-4ae8-4ebf-9308-a4b8bc52ae51 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:54.699774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:54.748983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:54.749662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.749998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.750229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.752619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:54.754099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:54.756976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 80/160] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:54 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:54.764718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.857832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03c4c-0ca7-40a5-b8ca-1008bff829b5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:54.869052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:54.918846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:54.919382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:54.919906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:54.922906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:54.923625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:54.927201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:54.928684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:54.942585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 81/161] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:54.948122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:54.955115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d09f3-a849-4d24-9ee3-21a242dae123 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:55.023703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:55.024324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:55.024604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:55.024876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:55.116198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:55.146939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:55.148678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:55.150240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5bf2f-0dc9-4293-8ca8-2be0ca859cc4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:55.212841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:55.213938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:55.215252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:55.215741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] PATH_INFO: `/v3/users/818c3ae70a684477af4280e7bbdcea2d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:55.217091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] RBAC: Authorizing `identity:get_user(user_id=818c3ae70a684477af4280e7bbdcea2d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:55.218988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:55.224718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 82/162] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:54 2022] GET /identity/v3/users/818c3ae70a684477af4280e7bbdcea2d => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:55.225769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:55.309017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:55.328236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 81/163] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:55.329117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:55.427268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e663-9a2f-46de-aef6-7fd5cf782577 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:55.496240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:55.497303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:55.497782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:55.498259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:55.499419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:55.517727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:55.522707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:55.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:55.524365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:55.532677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 83/164] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:30:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:55.533536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:55.553400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787227e8-4230-43f1-b0be-52ee3df075ba None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:55.632712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:55.633339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:55.633617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:55.633888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:55.735163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:55.735547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:55.736697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:55.745627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9abe64-6f5d-4f54-a0a9-929c11ec2e91 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:55.839871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:55.851314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:55.852031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:55.852317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:55.852591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:55.883819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 82/165] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:55.885948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:55.888129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:55.894240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:55.895137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:55.895928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 84/166] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:55.904548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.050590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d86cb2-33f9-4910-8fcc-b8ecad22660f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.114355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca65993-0055-453b-9dd7-76c6454bcb6c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.117188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.118022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.118446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.118871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] PATH_INFO: `/v3/projects/586e6f4a4108441a8b41cedf180e3540` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.120903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] RBAC: Authorizing `identity:get_project(project_id=586e6f4a4108441a8b41cedf180e3540)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.122527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.129640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 83/167] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:55 2022] GET /identity/v3/projects/586e6f4a4108441a8b41cedf180e3540 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.130578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.183175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.184417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.184839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.185278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.186087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:56.187285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.188839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.197615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 85/168] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:30:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.198708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.211789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:56.258899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:56.265862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645e622-b3dd-45c1-8cac-e06eb83a3c77 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.300776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7365804-19cd-4aca-9e95-2b00ef1f13ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.301419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7365804-19cd-4aca-9e95-2b00ef1f13ff None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.301704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7365804-19cd-4aca-9e95-2b00ef1f13ff None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.301976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7365804-19cd-4aca-9e95-2b00ef1f13ff None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.307447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7365804-19cd-4aca-9e95-2b00ef1f13ff None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.307447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7365804-19cd-4aca-9e95-2b00ef1f13ff None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.307447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 84/169] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.312572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616c24b-378e-415a-8b0d-916495f2589b None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.312858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616c24b-378e-415a-8b0d-916495f2589b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.313129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616c24b-378e-415a-8b0d-916495f2589b None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.314600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 85/170] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:56.315328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616c24b-378e-415a-8b0d-916495f2589b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.377565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e7f98-3d5f-48bd-8d1c-4a08db96dd33 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.416427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.417399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.417857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.418360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.419419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:56.420262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.422553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.432753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 86/171] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.433827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.487565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616c24b-378e-415a-8b0d-916495f2589b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.503772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:56.525786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.527356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.528801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.529125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.529895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=586e6f4a4108441a8b41cedf180e3540, user.id=475b2f85e68d43ac91521a15a826e5e3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:56.530200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.533531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.549824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 86/172] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/role_assignments?user.id=475b2f85e68d43ac91521a15a826e5e3&scope.project.id=586e6f4a4108441a8b41cedf180e3540&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.550700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.646228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:56.688103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289370d-2c81-46d0-80b8-af836afabba8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.692008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e0ee0-7dfb-406a-958c-671c0d8c7ad5 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.736038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.736680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.736961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.737236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.737970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=818c3ae70a684477af4280e7bbdcea2d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:56.738272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.739768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e36248-f9e6-4bff-8bf9-9ade8058806a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.745471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.746345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.746503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 87/173] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/role_assignments?user.id=818c3ae70a684477af4280e7bbdcea2d&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.746955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.747386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.748891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.751179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.751900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf22d8d-ee2c-4c33-b33c-d882ca1ecc7a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.752192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf22d8d-ee2c-4c33-b33c-d882ca1ecc7a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.752458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf22d8d-ee2c-4c33-b33c-d882ca1ecc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.753507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 87/174] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.754480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.780529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2cf22d8d-ee2c-4c33-b33c-d882ca1ecc7a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:56.783882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2cf22d8d-ee2c-4c33-b33c-d882ca1ecc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:56.825311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:56.874365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 88/175] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:56.876477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a3b8-0fa2-46b3-9464-90f3f6b894de None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.876754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a3b8-0fa2-46b3-9464-90f3f6b894de None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.877027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a3b8-0fa2-46b3-9464-90f3f6b894de None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.879592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 89/176] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:56.883510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246196bb-a22a-44e6-9525-f1b5e8e6da5d None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.883510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246196bb-a22a-44e6-9525-f1b5e8e6da5d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.883510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246196bb-a22a-44e6-9525-f1b5e8e6da5d None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.887286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 90/177] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:56.888317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246196bb-a22a-44e6-9525-f1b5e8e6da5d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:56.890696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6991090-b794-4ee3-8e9b-63cd3c9c6a6d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:56.936416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fcb52-95bc-42a8-9747-4b4994f7b97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:56.937506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fcb52-95bc-42a8-9747-4b4994f7b97c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.937990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fcb52-95bc-42a8-9747-4b4994f7b97c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.938486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fcb52-95bc-42a8-9747-4b4994f7b97c None admin] PATH_INFO: `/v3/users/475b2f85e68d43ac91521a15a826e5e3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.940013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fcb52-95bc-42a8-9747-4b4994f7b97c None admin] RBAC: Authorizing `identity:get_user(user_id=475b2f85e68d43ac91521a15a826e5e3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:56.941777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fcb52-95bc-42a8-9747-4b4994f7b97c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:56.943778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 88/178] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/users/475b2f85e68d43ac91521a15a826e5e3 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:56.945326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a853c4-9240-434a-b73f-7c548682173d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:56.945811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a853c4-9240-434a-b73f-7c548682173d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:56.946370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a853c4-9240-434a-b73f-7c548682173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:56.968140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-55a853c4-9240-434a-b73f-7c548682173d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:56.970605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-55a853c4-9240-434a-b73f-7c548682173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:56.990926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.033190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246196bb-a22a-44e6-9525-f1b5e8e6da5d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:57.082552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 89/179] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:57.083990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.084352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.084631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.084929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:57.086540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.086818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.087084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.106808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:57.108024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:57.144512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.183134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:57.185150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:57.189939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 90/180] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:57.193349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.391820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.416394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cd1dd-2b85-4370-99c3-0906915e295b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:57.416748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 91/181] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:57.417288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.457142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:57.457674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.457948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.458730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.489132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.521565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb05b-9a15-404e-bd29-2ee21a15f06a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:57.536808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:57.538270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:57.558340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c16755-4705-45bf-81c8-78ca1a5f7966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:57.559388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16755-4705-45bf-81c8-78ca1a5f7966 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.559898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16755-4705-45bf-81c8-78ca1a5f7966 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.560390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16755-4705-45bf-81c8-78ca1a5f7966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.561800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c16755-4705-45bf-81c8-78ca1a5f7966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:57.564142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c16755-4705-45bf-81c8-78ca1a5f7966 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:57.566170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 92/182] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:30:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:57.567507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf541-b380-44c0-85ee-de091bed84e1 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.568144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf541-b380-44c0-85ee-de091bed84e1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.568768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf541-b380-44c0-85ee-de091bed84e1 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.570248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 93/183] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:30:57.571242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cf541-b380-44c0-85ee-de091bed84e1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.618956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.641621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 91/184] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:57.642431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.672928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cf541-b380-44c0-85ee-de091bed84e1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:57.707867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:57.708436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.708706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.708971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.710102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:57.711574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:57.713198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 94/185] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:57 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:57.713931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.724146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.743718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ab372-2b92-4f61-9505-78bf9e201004 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:57.777615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ae23e-04ea-41c0-8314-8f79222b9dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:57.778230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ae23e-04ea-41c0-8314-8f79222b9dee None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.778503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ae23e-04ea-41c0-8314-8f79222b9dee None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.778764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ae23e-04ea-41c0-8314-8f79222b9dee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.790352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ae23e-04ea-41c0-8314-8f79222b9dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-197ae23e-04ea-41c0-8314-8f79222b9dee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:57.795177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:57.796253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:57.797539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 92/186] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:57.798195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.798195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.803183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.811836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eadb5-6525-442a-89c3-2429b6773b82 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:57.817858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:30:57.819037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:30:57.845230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:57.845829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:57.846101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:57.846363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:57.847001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:57.848451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:57.848945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:57.869355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 95/187] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:30:57 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:30:57.870164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.899471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 93/188] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:30:57.899471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:57.984329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.007890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6ce90-c60a-4146-b533-bd9cae8d40ea None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.043258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15e3d-c1ab-40b6-b500-bac27dea4fac None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.054581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.055521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.055808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.056075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.057245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.058702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.060596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 94/189] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:57 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.061217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.094224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.095150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.095473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.095895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.105903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.113090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:58.120827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:58.121701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:30:58.122491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:30:58.125182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 96/190] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:30:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:58.126312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.198805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e387632-a45d-41a2-a728-6fa13065058b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.252770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.253448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.253567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.253810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.254542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] RBAC: Adding query filter params (role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=475b2f85e68d43ac91521a15a826e5e3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:58.254877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.256321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.257187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8b8c-fea6-4b95-ab6c-269babaa866d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.264333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 95/191] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/role_assignments?user.id=475b2f85e68d43ac91521a15a826e5e3&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.265989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.287158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.301302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.302417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.302898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.303401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.304991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:58.305767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.307516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.317720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 97/192] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.319043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.453328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.465508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0b55b-ee6d-48e7-8123-ad1f60f68815 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.495463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce4410-f4dd-4950-8894-7325906daad9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.544880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.545411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.545720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.547037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.547981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=475b2f85e68d43ac91521a15a826e5e3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:58.548428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.549945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.553036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.554287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.554569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.555130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.556091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:58.556398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.557341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 96/193] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/role_assignments?user.id=475b2f85e68d43ac91521a15a826e5e3&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.558116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.558224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.570393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 98/194] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.571266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.658327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.761166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060bef-12c2-44b8-a8b2-d84ec8360d1c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.769882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14820b7c-c571-41f2-9abc-8e82319e8d55 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.804539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.805106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.805695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.805976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.813040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.813693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.813972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.814245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] PATH_INFO: `/v3/users/6a92b19d22d74f85bd94168500b9d6a8` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.818758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:30:58.824370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:30:58.825243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:30:58.826457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 97/195] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:30:58.827127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.834540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] RBAC: Authorizing `identity:get_user(user_id=6a92b19d22d74f85bd94168500b9d6a8)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.836315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.838206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 99/196] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/users/6a92b19d22d74f85bd94168500b9d6a8 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.839005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:58.857419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.944702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70809-6495-4cff-8232-e6c00eff4e46 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.968072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ba1a0-bbe5-478e-b82d-73e394895467 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:58.981477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:58.982180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:58.982460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:58.982775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] PATH_INFO: `/v3/users/6a92b19d22d74f85bd94168500b9d6a8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:58.984549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] RBAC: Authorizing `identity:get_user(user_id=6a92b19d22d74f85bd94168500b9d6a8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:58.984682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:58.986600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:58.988714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 98/197] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/users/6a92b19d22d74f85bd94168500b9d6a8 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:58.989530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.011122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.012193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.012650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.013181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.014048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:59.014527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.016172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.027208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 100/198] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:59.028404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.094615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.104553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d890e-9cb1-4d32-bbb2-679e69e66e57 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.150912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.151659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.151944 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.152212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.175730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5daa1-a9cc-4b68-8aff-15709ed994cb None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.217141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.217810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.218830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.219312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.234383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.236193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.273148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.323951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.326863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.349079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.360836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 99/199] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:59.361493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.444078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 101/200] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:59.444919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.504130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9756e-0208-45fd-96b4-f3a632e2292c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.517413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.557451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89c1ad-e23e-483d-a4c7-137a844bbe59 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.562587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.563505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.564011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.564524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] PATH_INFO: `/v3/users/bf1ee5df3590494e83b5a4c817f5d0e5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.566401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] RBAC: Authorizing `identity:get_user(user_id=bf1ee5df3590494e83b5a4c817f5d0e5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.568582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.571417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 100/201] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/users/bf1ee5df3590494e83b5a4c817f5d0e5 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:59.572422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.599187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.599808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.600086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.600355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.601481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.603270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.604597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 102/202] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:59.605433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.655511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.712327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a77ec-bf98-4ad4-b9fc-78ad88684105 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.714251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138bf23-5b3b-4776-9e36-09b1dd696884 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.748900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.749599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.749880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.750199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.754539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.762918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.763845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.765670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.766176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.767981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.770469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.773071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 101/203] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:59.774011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.837944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.840436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.885527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e390-d8f3-4f93-93c7-72ce29ed6da9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:30:59.914166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:30:59.924283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:30:59.925139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:30:59.925623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:30:59.926073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:30:59.927002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=6a92b19d22d74f85bd94168500b9d6a8) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:30:59.927619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:30:59.929280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:30:59.935645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 102/204] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/role_assignments?user.id=6a92b19d22d74f85bd94168500b9d6a8&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:30:59.942451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:30:59.965481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 103/205] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:30:59.972592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:00.017002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:00.104416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbda07e-c303-4c38-86b3-3887a7785c31 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:00.146977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:00.148001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.148496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.148976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.150849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:00.152571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:00.155034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 104/206] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:00.173488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:00.197545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b53d9-6825-4650-a687-95e0b70e3cf8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:00.248569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:00.249226 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.250040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.250427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.251385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] RBAC: Adding query filter params (role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=6a92b19d22d74f85bd94168500b9d6a8) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:00.251828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:00.253404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae79f01-28fb-4563-a317-a7f41e120509 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:00.260209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 103/207] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:30:59 2022] GET /identity/v3/role_assignments?user.id=6a92b19d22d74f85bd94168500b9d6a8&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:00.327932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f2b9a-e806-445f-9397-b45452348c4d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:00.330968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:00.386167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:00.386808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.387094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.387365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.388163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] RBAC: Adding query filter params (role.id=4bad7209656f49bfbc3d5d94c8824368, scope.project.id=d98f31b12d634fe4849900dfd66e2969, user.id=bf1ee5df3590494e83b5a4c817f5d0e5) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:00.388472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:00.390056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05314380-aec2-4de4-8ac3-fade7380a43a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:00.397942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 105/208] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:00 2022] GET /identity/v3/role_assignments?user.id=bf1ee5df3590494e83b5a4c817f5d0e5&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=4bad7209656f49bfbc3d5d94c8824368 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:00.514549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f6be2-0d92-42ae-8367-c6c35b2431dc None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.514549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f6be2-0d92-42ae-8367-c6c35b2431dc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.514549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f6be2-0d92-42ae-8367-c6c35b2431dc None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.514549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 104/209] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:00.521060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80538fd9-dd04-4c49-a6dd-426519394e9d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.521570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80538fd9-dd04-4c49-a6dd-426519394e9d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.521671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80538fd9-dd04-4c49-a6dd-426519394e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.543382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-80538fd9-dd04-4c49-a6dd-426519394e9d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:00.544657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-80538fd9-dd04-4c49-a6dd-426519394e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:00.632398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 106/210] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:00.717603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12161f-c591-4da3-a1e7-acdabfc79249 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.717906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12161f-c591-4da3-a1e7-acdabfc79249 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.718707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12161f-c591-4da3-a1e7-acdabfc79249 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.720359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 105/211] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:00.737688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:00.737965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:00.738239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:00.761750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:00.762936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:00.840982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 107/212] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:00.947035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12161f-c591-4da3-a1e7-acdabfc79249 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.014961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:01.065199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12161f-c591-4da3-a1e7-acdabfc79249 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:01.101563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:01.102199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:01.102478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:01.102749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:01.113159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:01.118465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:01.119342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:01.120158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 106/213] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:01.123796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.125709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.154016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:01.235343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2fd5-3aa4-43e9-b68a-6f46ce67e2b8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:01.253538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb105-b77c-4987-9a24-4e83b436077c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:01.275758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:01.277416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:01.278072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:01.278329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:01.278597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:01.282610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:01.284104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:01.285701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 108/214] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:01.290974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.301551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:01.302166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:01.302834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:01.303106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:01.303833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:01.304464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:01.306353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:01.319517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 107/215] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:01.323934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.402225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:01.409597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea657c77-f089-4c09-9bf4-eabe3d2c0014 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:01.446066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:01.446684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:01.446958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:01.447225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:01.447950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:01.449393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:01.457979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5246-f8e4-4702-92b8-e17743486510 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:01.475599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 109/216] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:01 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:01.498181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:01.499235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:01.499803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:01.500295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] PATH_INFO: `/v3/users/e9f969cc6cec43d38fe5d8952187faae` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:01.521806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] RBAC: Authorizing `identity:get_user(user_id=e9f969cc6cec43d38fe5d8952187faae)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:01.523660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:01.525598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 108/217] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:01 2022] GET /identity/v3/users/e9f969cc6cec43d38fe5d8952187faae => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:01.532081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.648391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65063fd0-5615-479d-94ef-383274f01731 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:01.687357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:01.688000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:01.688274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:01.688542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:01.711144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:01.760702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:01.762163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:01.873533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 110/218] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:01.879950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:01.927942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:02.006077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7d13f-4f40-4101-91db-8288c53faa75 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:02.045237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab109fb-66c0-4fcb-b471-82e8c9f9dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:02.045885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab109fb-66c0-4fcb-b471-82e8c9f9dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:02.046744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab109fb-66c0-4fcb-b471-82e8c9f9dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:02.047018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab109fb-66c0-4fcb-b471-82e8c9f9dfa6 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:02.048209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab109fb-66c0-4fcb-b471-82e8c9f9dfa6 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:02.049670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab109fb-66c0-4fcb-b471-82e8c9f9dfa6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:02.051507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 109/219] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:01 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:02.057557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:02.116742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:02.168061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca467e-dfb7-461e-a92d-2ad639a75c6a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:02.205065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:02.205681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:02.205961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:02.206231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:02.206968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=e9f969cc6cec43d38fe5d8952187faae) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:02.207277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:02.208806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417888e8-067d-48a9-bbc2-4d458d85994f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:02.214652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 111/220] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:02 2022] GET /identity/v3/role_assignments?user.id=e9f969cc6cec43d38fe5d8952187faae&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:02.519963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa21c2-4185-49e7-8cfa-f913d1fd8cc0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:02.520460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa21c2-4185-49e7-8cfa-f913d1fd8cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:02.520460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa21c2-4185-49e7-8cfa-f913d1fd8cc0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:02.521994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 110/221] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:02.540181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6efa8-4f4d-45c1-a742-301860e66013 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:02.540691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6efa8-4f4d-45c1-a742-301860e66013 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:02.540781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6efa8-4f4d-45c1-a742-301860e66013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:02.564783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0ed6efa8-4f4d-45c1-a742-301860e66013 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:02.565975 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0ed6efa8-4f4d-45c1-a742-301860e66013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:02.668819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 112/222] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:02.716884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf33965-2cab-4d58-b7c8-5f720afa3067 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:02.717299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf33965-2cab-4d58-b7c8-5f720afa3067 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:02.717761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf33965-2cab-4d58-b7c8-5f720afa3067 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:02.721030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 111/223] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:02.726905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:02.727177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:02.727470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:02.748142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:02.749418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:02.816636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:02.850694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 113/224] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:03.019056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf33965-2cab-4d58-b7c8-5f720afa3067 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.165069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf33965-2cab-4d58-b7c8-5f720afa3067 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:03.167813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:03.217053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:03.217699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:03.218005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:03.218279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:03.233346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:03.238268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:03.239170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:03.240007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:03.241084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 112/225] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:03.244518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.246346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.355732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb715b0-98ab-4ae0-ba52-2c0bce7b7635 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:03.412685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b799-6cf8-416b-87c2-14e2b1178cac None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:03.415152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:03.415728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:03.416000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:03.416267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:03.416928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:03.417220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:03.418600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:03.438364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 113/226] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:03.445953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.509967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:03.510381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:03.511151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:03.511330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:03.511589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:03.535351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:03.540259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:03.541580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:03.542383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:03.553867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 114/227] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:03.554845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.628662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:03.720591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959fc7e-7fe6-4be7-84c3-fd463ca00aed None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:03.749538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b32166-cbae-4366-aae3-2edce1356038 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:03.787103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:03.787817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:03.788095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:03.788366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] PATH_INFO: `/v3/users/6a92b19d22d74f85bd94168500b9d6a8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:03.790661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] RBAC: Authorizing `identity:get_user(user_id=6a92b19d22d74f85bd94168500b9d6a8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:03.793010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:03.795296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 114/228] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/users/6a92b19d22d74f85bd94168500b9d6a8 => generated 283 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:03.801675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.821262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:03.821880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:03.822145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:03.822414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:03.823146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:03.823464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:03.824796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:03.844127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 115/229] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:03.850626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:03.994351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:04.004614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8926d-87cf-48e3-9276-774c8d54c791 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:04.047571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:04.048222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.048501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.048767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.051709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:04.052325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:04.053581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:04.071525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 115/230] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:04.072464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:04.072974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2343a4-bf2c-4164-ac43-3def6c0a5bf7 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:04.168341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:04.169410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.169874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.170337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] PATH_INFO: `/v3/users/6a92b19d22d74f85bd94168500b9d6a8` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.175053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] RBAC: Authorizing `identity:get_user(user_id=6a92b19d22d74f85bd94168500b9d6a8)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:04.177316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:04.179877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 116/231] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:03 2022] GET /identity/v3/users/6a92b19d22d74f85bd94168500b9d6a8 => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:04.186109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:04.321338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d44de7-239b-4a7f-9195-6320922c8217 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:04.364089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e15d7-6fcd-46da-9d11-3f9d6059830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:04.365149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e15d7-6fcd-46da-9d11-3f9d6059830a None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.365643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e15d7-6fcd-46da-9d11-3f9d6059830a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.365924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e15d7-6fcd-46da-9d11-3f9d6059830a None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/6a92b19d22d74f85bd94168500b9d6a8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.369069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e15d7-6fcd-46da-9d11-3f9d6059830a None admin] RBAC: Authorizing `identity:create_grant(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a, user_id=6a92b19d22d74f85bd94168500b9d6a8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:04.370795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e15d7-6fcd-46da-9d11-3f9d6059830a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:04.391748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 116/232] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:04 2022] PUT /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/6a92b19d22d74f85bd94168500b9d6a8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:04.393198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a3904-e88a-4d67-a961-0b0d20992f34 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.393529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a3904-e88a-4d67-a961-0b0d20992f34 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.393813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a3904-e88a-4d67-a961-0b0d20992f34 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.395177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 117/233] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:04.406701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2aed4-eb85-4ec7-b5dc-e033129c238f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:04.410901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1cc2c-90e5-484a-b6f0-9ab6aa35879c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.411236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1cc2c-90e5-484a-b6f0-9ab6aa35879c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.411537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1cc2c-90e5-484a-b6f0-9ab6aa35879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.454707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-72b1cc2c-90e5-484a-b6f0-9ab6aa35879c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:04.456117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-72b1cc2c-90e5-484a-b6f0-9ab6aa35879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:04.502353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:04.502563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:04.503566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.504226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.504710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.507083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:04.508934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:04.516932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 117/234] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:04 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:04.524283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:04.599073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:04.615338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 118/235] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:04.676587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb4686-e171-4bb4-ac96-d83e0c5b8313 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:04.694921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d689725-4083-4d7e-8b9e-4a92986d6d04 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.695209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d689725-4083-4d7e-8b9e-4a92986d6d04 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.695476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d689725-4083-4d7e-8b9e-4a92986d6d04 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.697105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 119/236] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:04.704912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72481eca-b196-4983-9f73-218e76258385 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.705199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72481eca-b196-4983-9f73-218e76258385 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.705476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72481eca-b196-4983-9f73-218e76258385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.720957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:04.721995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:04.722473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:04.722933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/6a92b19d22d74f85bd94168500b9d6a8/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:04.726267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] RBAC: Authorizing `identity:create_grant(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a, user_id=6a92b19d22d74f85bd94168500b9d6a8, role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:04.731460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:04.739965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-72481eca-b196-4983-9f73-218e76258385 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:04.741231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-72481eca-b196-4983-9f73-218e76258385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:04.751138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 118/237] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:04 2022] PUT /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/6a92b19d22d74f85bd94168500b9d6a8/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:31:04.859482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 120/238] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:04.960400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.089201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:05.098365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefa9a3-8e1e-45f8-9820-0cb8f0c0030f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:05.135344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:05.135979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.136255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.136524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.148110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:05.153578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:05.154448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:05.156008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 119/239] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:05.161436 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72481eca-b196-4983-9f73-218e76258385 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.257842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.293350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:05.315615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72481eca-b196-4983-9f73-218e76258385 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:05.361300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:05.361932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.362204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.362468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.363591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:05.363897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:05.365533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b67fe9-2112-4ba5-8e6a-c192fa1e8c1d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:05.378110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 121/240] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:05.380043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a1155-92da-4904-b93b-1cf70f3da57b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.382726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a1155-92da-4904-b93b-1cf70f3da57b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.383011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a1155-92da-4904-b93b-1cf70f3da57b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.384361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 122/241] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:05.389051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a1155-92da-4904-b93b-1cf70f3da57b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.429684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd3146-26d7-4ae2-805a-34f33600fceb None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:05.479252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffe370-8284-4fdb-9775-220b6e560bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:05.480517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffe370-8284-4fdb-9775-220b6e560bde None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.481354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffe370-8284-4fdb-9775-220b6e560bde None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.481800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffe370-8284-4fdb-9775-220b6e560bde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.484898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffe370-8284-4fdb-9775-220b6e560bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:05.487011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffe370-8284-4fdb-9775-220b6e560bde None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:05.490499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 120/242] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:05.492074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.492518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.492942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.505992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a1155-92da-4904-b93b-1cf70f3da57b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:05.518028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:05.520153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:05.542650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:05.544152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:05.544940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.545299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.545637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] PATH_INFO: `/v3/users/0ba81a6b16f240f185c5cca86d5d6ac0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.560931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba81a6b16f240f185c5cca86d5d6ac0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:05.563154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:05.565112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 123/243] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:05 2022] GET /identity/v3/users/0ba81a6b16f240f185c5cca86d5d6ac0 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:05.565875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.568809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:05.615450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 121/244] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:05.615450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.742512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf54dae-ee5e-4144-8e95-f818cbe3f592 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:05.802885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958f9e-eee7-4e10-884d-6d248367f39b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:05.821104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:05.821716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.821988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.822253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.822908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:05.824360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:05.852916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 124/245] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:05 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:05.864191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:05.865132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:05.865548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:05.866030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:05.891553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:05.947339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:05.967008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:05.969899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:06.083088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:06.151225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18410cfb-c832-4cfd-9d59-f4b11b557ca6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.169746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 122/246] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:06.176222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:06.198740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.199733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.200201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.200607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.213257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:06.219895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:06.220814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:06.221589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:06.223583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 125/247] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:06.229295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:06.253643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:06.320512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93a4d-7718-49fe-8be5-bffbb2507fc1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.342186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a530405-52a6-40c8-99fe-d15b718ee9d0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.361921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:06.370902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.371580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.371855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.372117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.373935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:06.376270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:06.379089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 123/248] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:06 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:06.384867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:06.391047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.391999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.392478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.392938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.393813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:06.394347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:06.396014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:06.406314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 126/249] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:06.418534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:06.543392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560bd6-1b34-4358-bb1c-f78ba44a659f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.563251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a6485-8bb1-4ab6-8fc5-89aafc257a24 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.575587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:06.601133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.602082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.602717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.603852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.604849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] RBAC: Adding query filter params (role.id=4bad7209656f49bfbc3d5d94c8824368, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=0ba81a6b16f240f185c5cca86d5d6ac0) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:06.605158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:06.607491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:06.612150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.613037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.613536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.614013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] PATH_INFO: `/v3/users/e9f969cc6cec43d38fe5d8952187faae` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.614260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 124/250] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:06 2022] GET /identity/v3/role_assignments?user.id=0ba81a6b16f240f185c5cca86d5d6ac0&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=4bad7209656f49bfbc3d5d94c8824368 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:06.615667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] RBAC: Authorizing `identity:get_user(user_id=e9f969cc6cec43d38fe5d8952187faae)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:06.617341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:06.619602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 127/251] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:06 2022] GET /identity/v3/users/e9f969cc6cec43d38fe5d8952187faae => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:06.626226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:06.749032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b27c7-9039-4565-8a04-71771311bd21 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.794677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad9ec8-a526-43c8-8316-1bbc6977dca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.795338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad9ec8-a526-43c8-8316-1bbc6977dca1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.795638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad9ec8-a526-43c8-8316-1bbc6977dca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.796432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad9ec8-a526-43c8-8316-1bbc6977dca1 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.797600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad9ec8-a526-43c8-8316-1bbc6977dca1 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:06.799080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad9ec8-a526-43c8-8316-1bbc6977dca1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:06.801254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 125/252] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:06 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:06.807212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:06.884684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:06.949034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb23ca6-57bb-4b30-95f5-5a275d3b5e34 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:06.958554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc76cc-5f9f-4ed2-ba6a-311c3db98792 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.958850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc76cc-5f9f-4ed2-ba6a-311c3db98792 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.959120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc76cc-5f9f-4ed2-ba6a-311c3db98792 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.960490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 126/253] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:06.973513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527567-a198-4e99-8703-00d51c85ae12 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.973798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527567-a198-4e99-8703-00d51c85ae12 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.974381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527567-a198-4e99-8703-00d51c85ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.992500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f2ff5-b38e-4ea1-92af-2d6dcb5e30c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:06.993418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f2ff5-b38e-4ea1-92af-2d6dcb5e30c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:06.993855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f2ff5-b38e-4ea1-92af-2d6dcb5e30c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:06.994270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f2ff5-b38e-4ea1-92af-2d6dcb5e30c7 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/e9f969cc6cec43d38fe5d8952187faae/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:06.996456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f2ff5-b38e-4ea1-92af-2d6dcb5e30c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a, user_id=e9f969cc6cec43d38fe5d8952187faae, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:06.998080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f2ff5-b38e-4ea1-92af-2d6dcb5e30c7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:07.009381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4a527567-a198-4e99-8703-00d51c85ae12 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:07.010857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4a527567-a198-4e99-8703-00d51c85ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:07.020916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 128/254] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:06 2022] PUT /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/e9f969cc6cec43d38fe5d8952187faae/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:07.085671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:07.117717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 127/255] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:07.446150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de3cdc-7868-479e-a64e-e30374b680fb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:07.446826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de3cdc-7868-479e-a64e-e30374b680fb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:07.447295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de3cdc-7868-479e-a64e-e30374b680fb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:07.449122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 129/256] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:07.454247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:07.454753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:07.455212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:07.477896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:07.479437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:07.562325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 128/257] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:07.678378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de3cdc-7868-479e-a64e-e30374b680fb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:07.873680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de3cdc-7868-479e-a64e-e30374b680fb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:07.898057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:07.925917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:07.926814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:07.927234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:07.927661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:07.945107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:07.955452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:07.956424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:07.957195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:07.964914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 130/258] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:07.971570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:08.106264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaddefb-fd1c-46a0-8214-4e15f9b9210f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:08.131876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:08.162385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:08.163011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:08.163286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:08.163577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:08.164244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:08.164542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:08.165935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:08.179315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 129/259] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:08.185593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:08.250839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:08.429471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f67b7d-24d5-4def-80a5-fee551e2d477 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:08.429677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17e1b6-4ac2-4068-9a19-3f41c2b05493 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:08.469034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:08.470047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:08.470528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:08.471022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:08.483749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:08.489753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:08.490720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:08.491542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:08.494420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 130/260] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:08.502300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:08.520966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:08.521935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:08.522415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:08.522883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] PATH_INFO: `/v3/users/6a92b19d22d74f85bd94168500b9d6a8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:08.524315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] RBAC: Authorizing `identity:get_user(user_id=6a92b19d22d74f85bd94168500b9d6a8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:08.525980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:08.528118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 131/261] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:08 2022] GET /identity/v3/users/6a92b19d22d74f85bd94168500b9d6a8 => generated 283 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:08.534039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:08.593068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:08.619045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:08.693332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02b34-0334-428c-9dff-1603db09fd05 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:08.750052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:08.750911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:08.751339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:08.751776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:08.772601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106e69-b5e4-46ed-94f8-32802e77faad None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:08.832426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:08.833043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:08.833323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:08.833597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:08.834265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:08.834562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:08.836139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:08.845208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 131/262] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:08.852371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:08.896170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:08.898213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:08.958072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:09.003797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:09.017478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962694f-e269-4e85-a861-340dcae8cd5d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:09.063093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 132/263] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:09.074750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:09.085047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:09.085705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.085986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.086262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] PATH_INFO: `/v3/users/6a92b19d22d74f85bd94168500b9d6a8` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.092131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] RBAC: Authorizing `identity:get_user(user_id=6a92b19d22d74f85bd94168500b9d6a8)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:09.094797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:09.096496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 132/264] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:08 2022] GET /identity/v3/users/6a92b19d22d74f85bd94168500b9d6a8 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:09.104785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:09.257166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95150a-2657-4a2f-83c8-744b241a7ad8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:09.292314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:09.293189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.293634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.294042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.295388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:09.297044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:09.299142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 133/265] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:09 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:09.305292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:09.312208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:09.332271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43ff6-e27b-4881-9648-61ca089cec1c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:09.409840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:09.410469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.410745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.411012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.487563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4a453-69d1-444e-85e6-d084e1e5e756 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:09.508525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:09.523378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:09.524141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.524515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.524881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.525862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=6a92b19d22d74f85bd94168500b9d6a8) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:09.526286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:09.527873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b67446-2622-478a-b8e2-eebff0f34276 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:09.535185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 134/266] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:09 2022] GET /identity/v3/role_assignments?user.id=6a92b19d22d74f85bd94168500b9d6a8&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:09.554740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf09977-2630-4717-a069-b224e3ad5c4c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.555435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf09977-2630-4717-a069-b224e3ad5c4c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.555806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf09977-2630-4717-a069-b224e3ad5c4c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.557521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 135/267] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:09.563401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bfc7c-e7e3-471b-8029-1e7d412d9745 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.563847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bfc7c-e7e3-471b-8029-1e7d412d9745 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.564213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bfc7c-e7e3-471b-8029-1e7d412d9745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.584193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5b1bfc7c-e7e3-471b-8029-1e7d412d9745 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:09.585594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5b1bfc7c-e7e3-471b-8029-1e7d412d9745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:09.600606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:09.602384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:09.665296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 136/268] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:09.671448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:09.761489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f0897-6c8e-493b-a827-8e96e707d3d4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.761693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f0897-6c8e-493b-a827-8e96e707d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.762131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f0897-6c8e-493b-a827-8e96e707d3d4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.764454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 137/269] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:09.771871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:09.772509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:09.772928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:09.795866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:09.798397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:09.809901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 133/270] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:31:09.816462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:09.899923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 138/271] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:09.956258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:09.972726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.011022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21bde0-6a3b-4b64-967b-9b821f21f1fa None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.086284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.086922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.087195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.087481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.088780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:10.090246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:10.095273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 134/272] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:09 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:10.100834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.178459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51069b05-e9e8-43a5-af6d-bc7acb436cd2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.229423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:10.246341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.246979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.247253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.247608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.250337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:10.251913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:10.253634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 139/273] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:10.259205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.292155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b44bdc-34a0-4d81-a91d-81a32931bf84 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.338079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:10.345723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.346706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.347128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.347608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.348482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] RBAC: Adding query filter params (role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=6a92b19d22d74f85bd94168500b9d6a8) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:10.348935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:10.350646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:10.357563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 135/274] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:10 2022] GET /identity/v3/role_assignments?user.id=6a92b19d22d74f85bd94168500b9d6a8&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:10.359731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.385364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b0f77-c190-41dd-a43d-6e1c4d79d179 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.451384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89cf79-b3f5-42a0-8210-96ec809314be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.452072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89cf79-b3f5-42a0-8210-96ec809314be None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.452345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89cf79-b3f5-42a0-8210-96ec809314be None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.452610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89cf79-b3f5-42a0-8210-96ec809314be None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.453386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89cf79-b3f5-42a0-8210-96ec809314be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:10.454875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89cf79-b3f5-42a0-8210-96ec809314be None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:10.479496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 140/275] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:10 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:31:10.485557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b89743-e5ba-4af6-ae2e-a8929f0c21dd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.485890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b89743-e5ba-4af6-ae2e-a8929f0c21dd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.486149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b89743-e5ba-4af6-ae2e-a8929f0c21dd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.487703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 141/276] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:10.501198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4bf6-14e2-49cb-9939-084455027171 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.501543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4bf6-14e2-49cb-9939-084455027171 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.501746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4bf6-14e2-49cb-9939-084455027171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.534250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-23ee4bf6-14e2-49cb-9939-084455027171 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:10.535531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-23ee4bf6-14e2-49cb-9939-084455027171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:10.536189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7c811-78c7-4738-837d-bf30035943af None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.588404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc0690-dc41-42b7-a9b7-cd64f324484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.589003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0690-dc41-42b7-a9b7-cd64f324484a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.589283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0690-dc41-42b7-a9b7-cd64f324484a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.589558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0690-dc41-42b7-a9b7-cd64f324484a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.605998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc0690-dc41-42b7-a9b7-cd64f324484a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:10.607186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-9ebc0690-dc41-42b7-a9b7-cd64f324484a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:10.613805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:10.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:10.615747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 136/277] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:10.617264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e144d-3f7b-493b-8cb3-9e45be41987d None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.618278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e144d-3f7b-493b-8cb3-9e45be41987d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.618278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e144d-3f7b-493b-8cb3-9e45be41987d None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.619389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 137/278] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:10.625042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e144d-3f7b-493b-8cb3-9e45be41987d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.633617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 142/279] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:10.635123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.635401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.636197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.656601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:10.658177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:10.719271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:10.746554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 143/280] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:10.772509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e144d-3f7b-493b-8cb3-9e45be41987d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.810936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.814407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.814407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.814407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:10.814407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:10.814407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:10.815405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:10.824570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 138/281] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:10.831226 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.941797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:10.949310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1772072-b34e-43d7-a490-2c5cdc87a938 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:10.969475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:10.995672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:10.996447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:10.996725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:10.996992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] PATH_INFO: `/v3/users/a664525045f940b191b20d6ab98c91aa` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.012397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] RBAC: Authorizing `identity:get_user(user_id=a664525045f940b191b20d6ab98c91aa)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:11.014046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:11.016378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 144/282] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:10 2022] GET /identity/v3/users/a664525045f940b191b20d6ab98c91aa => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:11.022868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.093532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064070-d0fb-427d-88e9-c9d7fc04e9b4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.118499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:11.136769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.137552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.137922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.138285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.149999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:11.154978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:11.156289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:11.157380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 139/283] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:11.158028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.158662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a45f53-baf2-48d8-a894-3e9c37935e6e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.211269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.211982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.212261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.212531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.213569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, user.id=a664525045f940b191b20d6ab98c91aa, scope.system=all) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:11.213878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:11.216121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:11.223493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 145/284] 10.222.0.55 () {56 vars in 1308 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/role_assignments?user.id=a664525045f940b191b20d6ab98c91aa&scope.system=all&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:11.229772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.293343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff7ae9-30a5-477b-8d44-498421b79d89 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.307863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:11.340333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.341483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.341765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.342036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.361581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:11.366649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:11.368840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:11.370830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 140/285] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:11.380705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.433441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:11.457191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4887da-abbe-4838-ac70-a61ff1b5f253 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.510940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9099d-49da-4470-a71d-b4bedd739c24 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.524918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.525880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.526358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.526823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.527713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:11.528247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:11.529817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:11.547128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 146/286] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:11.554118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.586377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.587381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.587884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.588349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.589213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:11.590519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:11.592170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:11.605959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 141/287] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:11.612463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.747630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cabb7b-fc79-48ff-a8de-8e36bad95ac1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.751873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:11.755562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:11.777494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66337375-2f26-4c99-a4e6-f699f7e1bd39 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.791983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.792948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.793430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.793922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] PATH_INFO: `/v3/users/e9f969cc6cec43d38fe5d8952187faae` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.795396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] RBAC: Authorizing `identity:get_user(user_id=e9f969cc6cec43d38fe5d8952187faae)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:11.798178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:11.801087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 142/288] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/users/e9f969cc6cec43d38fe5d8952187faae => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:11.807957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.844038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.844680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.844961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.845230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] PATH_INFO: `/v3/users/0ba81a6b16f240f185c5cca86d5d6ac0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:11.846397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba81a6b16f240f185c5cca86d5d6ac0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:11.847913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:11.849496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 147/289] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/users/0ba81a6b16f240f185c5cca86d5d6ac0 => generated 297 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:11.854974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:11.920182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138cd0-9635-498d-91e6-e7b36e838271 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:11.957700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:11.958669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:11.959183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:11.959684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:12.023917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:12.036019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:12.037740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:12.161267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 143/290] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:31:12.164998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e592f-aa21-41a8-a2ff-c1b2b0453391 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:12.167952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:12.268134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:12.283288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:12.284355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:12.284847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:12.285315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:12.291770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:12.293162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:12.299610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 148/291] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:11 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:12.302292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97f61-15fb-479f-a4c7-448b605ae0be None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:12.303950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:12.350104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:12.350806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:12.351088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:12.351361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:12.355097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:12.360501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:12.363728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 144/292] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:12 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:12.375865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:12.393181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:12.532701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7b32-b546-48b2-bf5a-5ef48c787d63 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:12.551273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6e823-1f9e-4f73-997e-90d67d6695a2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:12.604035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:12.604735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:12.605015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:12.605182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ef293-a325-4a35-a4b2-e14dc6a234c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:12.605290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:12.605855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef293-a325-4a35-a4b2-e14dc6a234c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:12.606037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=e9f969cc6cec43d38fe5d8952187faae) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:12.606129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef293-a325-4a35-a4b2-e14dc6a234c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:12.606310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:12.606404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef293-a325-4a35-a4b2-e14dc6a234c5 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/0ba81a6b16f240f185c5cca86d5d6ac0/roles/4bad7209656f49bfbc3d5d94c8824368` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:12.607785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8728b-afeb-4a26-8510-c0f6618fdc22 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:12.611652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ef293-a325-4a35-a4b2-e14dc6a234c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a, user_id=0ba81a6b16f240f185c5cca86d5d6ac0, role_id=4bad7209656f49bfbc3d5d94c8824368)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:12.613102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ef293-a325-4a35-a4b2-e14dc6a234c5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:12.614554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 145/293] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:12 2022] GET /identity/v3/role_assignments?user.id=e9f969cc6cec43d38fe5d8952187faae&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:12.614883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:12.643256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 149/294] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:12 2022] PUT /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/users/0ba81a6b16f240f185c5cca86d5d6ac0/roles/4bad7209656f49bfbc3d5d94c8824368 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:13.233762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c803fd-b02d-4933-8c45-4768cd3ce710 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.235044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c803fd-b02d-4933-8c45-4768cd3ce710 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.235627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c803fd-b02d-4933-8c45-4768cd3ce710 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.237213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 146/295] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:13.243237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfeae6-2602-4c6a-afe4-deeb9656c938 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.244172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfeae6-2602-4c6a-afe4-deeb9656c938 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.244641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfeae6-2602-4c6a-afe4-deeb9656c938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.266729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-75dfeae6-2602-4c6a-afe4-deeb9656c938 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:13.268289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-75dfeae6-2602-4c6a-afe4-deeb9656c938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:13.348795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 150/296] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:13.538552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745134-8257-427f-aac6-d712c19f9209 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.539034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745134-8257-427f-aac6-d712c19f9209 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.539445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745134-8257-427f-aac6-d712c19f9209 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.540900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 147/297] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:13.546282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecbdc1-07bb-497d-93b3-a794522dd357 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.546682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecbdc1-07bb-497d-93b3-a794522dd357 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.547061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecbdc1-07bb-497d-93b3-a794522dd357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.567368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-88ecbdc1-07bb-497d-93b3-a794522dd357 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:13.568603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-88ecbdc1-07bb-497d-93b3-a794522dd357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:13.587169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:13.627202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745134-8257-427f-aac6-d712c19f9209 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:13.644913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 151/298] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:13.744911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745134-8257-427f-aac6-d712c19f9209 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:13.745753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addecac-038f-4e3f-9a52-09fa92b204d9 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.746036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addecac-038f-4e3f-9a52-09fa92b204d9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.746306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addecac-038f-4e3f-9a52-09fa92b204d9 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.747925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 152/299] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:13.753787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.754065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.754337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.774070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:13.775714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:13.786948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:13.787999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.788379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.788752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.801285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:13.807499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:13.808564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:13.809383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:13.815112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 148/300] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:13.816325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:13.855868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 153/301] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:13.884002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:13.926936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1320eb-383f-415c-8344-9428921a6f31 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:13.961278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:13.962141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:13.962968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:13.963413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:13.963874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:13.964758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:13.965344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:13.966921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:13.980871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 149/302] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:13.988847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.048530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:14.079740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029d91e-0212-40f9-86ea-e3c0dc4bfc84 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.115978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1f514-03d1-4c76-8200-532e0972eab7 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.117926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.118724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.119147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.119615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.120880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.122468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.129267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 154/303] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:14.130354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.143478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:14.157891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.158557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.158836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.159105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] PATH_INFO: `/v3/users/30ead664714f42b7906f1bde1ddaf9e1` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.172703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] RBAC: Authorizing `identity:get_user(user_id=30ead664714f42b7906f1bde1ddaf9e1)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.174181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.176145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 150/304] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:13 2022] GET /identity/v3/users/30ead664714f42b7906f1bde1ddaf9e1 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:14.176806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.254063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fd42c-eee9-4cbe-8dc9-612f65ff8f68 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.284371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ecbad-e099-413c-8831-dab94897cf30 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.291784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.292394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.292666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.292929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] PATH_INFO: `/v3/groups` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.293555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.294963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.327463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 155/305] 10.222.0.55 () {60 vars in 1134 bytes} [Sat Sep 24 19:31:14 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:14.327463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.332339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.332339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.332339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.332759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.343674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:14.345864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:14.350874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:14.352246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:14.353191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 151/306] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:14.358795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.417682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:14.480220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76b3a-89fa-42e0-9938-b2c20f9863c3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.505944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9497dc-3829-4fda-be6d-23f50891e5ae None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.534529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.535146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.535501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.535776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.536504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] RBAC: Adding query filter params (role.id=4bad7209656f49bfbc3d5d94c8824368, user.id=30ead664714f42b7906f1bde1ddaf9e1, scope.system=all) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:14.536807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.538268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.544322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 156/307] 10.222.0.55 () {56 vars in 1308 bytes} [Sat Sep 24 19:31:14 2022] GET /identity/v3/role_assignments?user.id=30ead664714f42b7906f1bde1ddaf9e1&scope.system=all&role.id=4bad7209656f49bfbc3d5d94c8824368 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:14.553372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.554419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.554913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.555676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.556409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:14.556716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.558165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.569009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 152/308] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:14.574889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.662648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:14.694994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b15afa-d27a-43c3-a870-cfee5469aaaa None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.735293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cba7fc-de3e-400b-a8c9-8cd6ffd3c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.736217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cba7fc-de3e-400b-a8c9-8cd6ffd3c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.736697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cba7fc-de3e-400b-a8c9-8cd6ffd3c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.737104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cba7fc-de3e-400b-a8c9-8cd6ffd3c5a8 None admin] PATH_INFO: `/v3/users/a664525045f940b191b20d6ab98c91aa` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.738400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cba7fc-de3e-400b-a8c9-8cd6ffd3c5a8 None admin] RBAC: Authorizing `identity:get_user(user_id=a664525045f940b191b20d6ab98c91aa)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.740091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cba7fc-de3e-400b-a8c9-8cd6ffd3c5a8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.741848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 157/309] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:14 2022] GET /identity/v3/users/a664525045f940b191b20d6ab98c91aa => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:14.747563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:14.854616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb749bd-b7a1-40ec-a969-e723ff84f122 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:14.856410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:14.891968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959b7ed-61de-4b4d-835e-ebefaa6a57c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:14.893066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b7ed-61de-4b4d-835e-ebefaa6a57c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:14.893482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b7ed-61de-4b4d-835e-ebefaa6a57c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:14.893944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b7ed-61de-4b4d-835e-ebefaa6a57c5 None admin] PATH_INFO: `/v3/system/users/a664525045f940b191b20d6ab98c91aa/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:14.895521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959b7ed-61de-4b4d-835e-ebefaa6a57c5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a664525045f940b191b20d6ab98c91aa, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:14.897032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959b7ed-61de-4b4d-835e-ebefaa6a57c5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:14.908973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 153/310] 10.222.0.55 () {58 vars in 1334 bytes} [Sat Sep 24 19:31:14 2022] PUT /identity/v3/system/users/a664525045f940b191b20d6ab98c91aa/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:15.073927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c9a49-97ce-4a36-ad00-264b0c354836 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:15.074252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c9a49-97ce-4a36-ad00-264b0c354836 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:15.074478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c9a49-97ce-4a36-ad00-264b0c354836 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:15.076010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 158/311] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:15.086061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:15.086061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:15.086061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:15.103264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:15.104738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:15.189353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 154/312] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:15.481138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c9a49-97ce-4a36-ad00-264b0c354836 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:15.587860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c9a49-97ce-4a36-ad00-264b0c354836 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:15.625683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:15.626593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:15.627012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:15.627444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:15.638373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:15.644857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:15.645985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:15.646964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:15.649209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 159/313] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:15.655979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:15.759197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:15.766270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785948be-1cb9-4bac-a58e-819719e53d7d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:15.802462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:15.803080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:15.803357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:15.803734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:15.804406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:15.804705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:15.806355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:15.815757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 155/314] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:15.821534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:15.830080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:15.944487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f1f7e-fbec-4cde-aa79-6baef9a56777 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:15.986968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:15.987691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:15.988012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:15.988287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] PATH_INFO: `/v3/users/0ba81a6b16f240f185c5cca86d5d6ac0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:15.989560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba81a6b16f240f185c5cca86d5d6ac0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:15.992522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:15.994445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 160/315] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:15 2022] GET /identity/v3/users/0ba81a6b16f240f185c5cca86d5d6ac0 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:16.000624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:16.134644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b6a33-aa48-41c6-9239-3fd2b24fa970 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:16.171690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:16.172487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.172762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.173030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.201717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:16.246466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:16.247971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:16.354514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 156/316] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:16.360564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:16.442673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:16.478645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa24d8-5b01-4d49-8753-9ed47a91cde0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:16.517957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775f2e4-f815-43af-ac0c-105afe621437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:16.518599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775f2e4-f815-43af-ac0c-105afe621437 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.518859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775f2e4-f815-43af-ac0c-105afe621437 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.519131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775f2e4-f815-43af-ac0c-105afe621437 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.520364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775f2e4-f815-43af-ac0c-105afe621437 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:16.522003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775f2e4-f815-43af-ac0c-105afe621437 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:16.524018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 161/317] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:16 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:16.531001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:16.596684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:16.648889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe80fb-bc0e-41b4-ba35-9c9ca894bb8b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:16.688431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:16.689104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.689315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.689586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.690323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] RBAC: Adding query filter params (role.id=4bad7209656f49bfbc3d5d94c8824368, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a, user.id=0ba81a6b16f240f185c5cca86d5d6ac0) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:16.690630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:16.692191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278055-9a12-40c0-89ef-016a1ed45acf None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:16.699027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 157/318] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:16 2022] GET /identity/v3/role_assignments?user.id=0ba81a6b16f240f185c5cca86d5d6ac0&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=4bad7209656f49bfbc3d5d94c8824368 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:16.751873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9036-8ddd-40f6-861e-46ed3707be10 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.752509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9036-8ddd-40f6-861e-46ed3707be10 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.753015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9036-8ddd-40f6-861e-46ed3707be10 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.754971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 162/319] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:16.761324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe955d-48d2-42b2-879c-54cf1c9aa1b8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.761660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe955d-48d2-42b2-879c-54cf1c9aa1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.761868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe955d-48d2-42b2-879c-54cf1c9aa1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.783324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-cafe955d-48d2-42b2-879c-54cf1c9aa1b8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:16.784654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cafe955d-48d2-42b2-879c-54cf1c9aa1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:16.867056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 158/320] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:16.889862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166cca3-2c4e-4755-b391-f793f8ed12c0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.890169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166cca3-2c4e-4755-b391-f793f8ed12c0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.890429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166cca3-2c4e-4755-b391-f793f8ed12c0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.893191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 163/321] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:16.897748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:16.898035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:16.898311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:16.918767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:16.920295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:16.980655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:17.013157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 159/322] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:17.147188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166cca3-2c4e-4755-b391-f793f8ed12c0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.258756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166cca3-2c4e-4755-b391-f793f8ed12c0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:17.294363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:17.294995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.295360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.295871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.296981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.297108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:17.298439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:17.300054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 164/323] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:17.305782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.316076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:17.397805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c049c7c-16c8-4e57-8d64-be735713189b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:17.407619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c452e9c-83b2-4a3e-a76a-ca527de026ea None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:17.431266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cb751-6cd6-492d-a654-d80740d3258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:17.431925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cb751-6cd6-492d-a654-d80740d3258f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.432198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cb751-6cd6-492d-a654-d80740d3258f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.432463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cb751-6cd6-492d-a654-d80740d3258f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.442028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8725458-28f2-42dc-b049-196df40d5685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:17.442623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8725458-28f2-42dc-b049-196df40d5685 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.442889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cb751-6cd6-492d-a654-d80740d3258f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:17.442982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8725458-28f2-42dc-b049-196df40d5685 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.443152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8725458-28f2-42dc-b049-196df40d5685 None admin] PATH_INFO: `/v3/groups` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.443818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8725458-28f2-42dc-b049-196df40d5685 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:17.445246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8725458-28f2-42dc-b049-196df40d5685 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-5b2cb751-6cd6-492d-a654-d80740d3258f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:17.447714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:17.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:17.449329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 160/324] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:17.450435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d30d7-8577-4621-85e7-3a5e8478e8e8 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.450528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d30d7-8577-4621-85e7-3a5e8478e8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.450761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d30d7-8577-4621-85e7-3a5e8478e8e8 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.452793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 161/325] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:17.456527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d30d7-8577-4621-85e7-3a5e8478e8e8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.464015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 165/326] 10.222.0.55 () {60 vars in 1134 bytes} [Sat Sep 24 19:31:17 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:31:17.465403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.465694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.465965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.486672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:17.487956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:17.524314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:17.566710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:17.587456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 166/327] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:17.610792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d30d7-8577-4621-85e7-3a5e8478e8e8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:17.652453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:17.653067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.653344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.653610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.654359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:17.654661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:17.656874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:17.666797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 162/328] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:17.673133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.802296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d302d2-b2a6-4414-a86c-dc4d401f7678 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:17.849348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:17.850001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:17.850279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:17.850547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] PATH_INFO: `/v3/users/30ead664714f42b7906f1bde1ddaf9e1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:17.851853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] RBAC: Authorizing `identity:get_user(user_id=30ead664714f42b7906f1bde1ddaf9e1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:17.853317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:17.855114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 167/329] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:17 2022] GET /identity/v3/users/30ead664714f42b7906f1bde1ddaf9e1 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:17.860438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.863632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:17.947114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:18.001640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba977a-a659-45a8-8dda-b9e0d3b35a73 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:18.005963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e695ca4-e936-46ba-9c6b-ca9d32dea304 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:18.035847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:18.043546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:18.044202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:18.044475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:18.044734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:18.044867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] PATH_INFO: `/v3/system/users/30ead664714f42b7906f1bde1ddaf9e1/roles/4bad7209656f49bfbc3d5d94c8824368` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:18.046702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=30ead664714f42b7906f1bde1ddaf9e1, role_id=4bad7209656f49bfbc3d5d94c8824368)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:18.048270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:18.054568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:18.055496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:18.056419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:18.063707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 168/330] 10.222.0.55 () {58 vars in 1334 bytes} [Sat Sep 24 19:31:17 2022] PUT /identity/v3/system/users/30ead664714f42b7906f1bde1ddaf9e1/roles/4bad7209656f49bfbc3d5d94c8824368 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:18.080984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:18.087168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:18.088256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:18.089558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 163/331] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:18.100666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:18.226352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55daf49-24cc-42df-9dc3-7ae047067b3b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:18.261800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:18.271544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:18.272602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:18.272929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:18.273377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:18.274068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:18.274375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:18.275851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:18.286255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 169/332] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:18.292030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:18.438441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786490c-0f9a-44ad-8e23-8098aa2fa288 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:18.487273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294542e-1c14-4593-865d-17cd8d7a2bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:18.487989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294542e-1c14-4593-865d-17cd8d7a2bda None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:18.488273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294542e-1c14-4593-865d-17cd8d7a2bda None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:18.488545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294542e-1c14-4593-865d-17cd8d7a2bda None admin] PATH_INFO: `/v3/users/a664525045f940b191b20d6ab98c91aa` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:18.489724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294542e-1c14-4593-865d-17cd8d7a2bda None admin] RBAC: Authorizing `identity:get_user(user_id=a664525045f940b191b20d6ab98c91aa)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:18.491185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294542e-1c14-4593-865d-17cd8d7a2bda None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:18.493111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 164/333] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:18 2022] GET /identity/v3/users/a664525045f940b191b20d6ab98c91aa => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:18.499683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:18.624513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9abaf-32a9-495c-a093-e887b15f83a1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:18.664071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:18.668274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:18.668569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:18.668847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:18.669592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] RBAC: Adding query filter params (role.id=1abc6940fcf2429aab6c6fc960488040, user.id=a664525045f940b191b20d6ab98c91aa, scope.system=all) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:18.669901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:18.672128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9beefeb-33cb-4be2-91dd-519c03c22a7d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:18.691247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 170/334] 10.222.0.55 () {56 vars in 1308 bytes} [Sat Sep 24 19:31:18 2022] GET /identity/v3/role_assignments?user.id=a664525045f940b191b20d6ab98c91aa&scope.system=all&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:20.107796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6129fa-ea43-4b70-ac71-ff80c4545d59 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:20.108311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6129fa-ea43-4b70-ac71-ff80c4545d59 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:20.108410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6129fa-ea43-4b70-ac71-ff80c4545d59 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:20.109699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 165/335] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:20.115241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:20.115708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:20.115804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:20.145266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:20.146463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:20.206356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:20.237757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 171/336] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:20.752835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6129fa-ea43-4b70-ac71-ff80c4545d59 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:20.794478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:20.875176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6129fa-ea43-4b70-ac71-ff80c4545d59 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:20.922146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed6d58-3b00-4592-84fe-9285ed044fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:20.922793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6d58-3b00-4592-84fe-9285ed044fec None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:20.923071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6d58-3b00-4592-84fe-9285ed044fec None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:20.923341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6d58-3b00-4592-84fe-9285ed044fec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:20.939071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed6d58-3b00-4592-84fe-9285ed044fec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-26ed6d58-3b00-4592-84fe-9285ed044fec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:20.944611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:20.945789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:20.946642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 166/337] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:20.954673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:21.075449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407a88b-2b09-4539-81b3-4e5acb9298d8 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.075449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407a88b-2b09-4539-81b3-4e5acb9298d8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.075449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407a88b-2b09-4539-81b3-4e5acb9298d8 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.075449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 167/338] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:21.092733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35509f0-e00c-415f-a86f-df9f68fabb32 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.093164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35509f0-e00c-415f-a86f-df9f68fabb32 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.093619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35509f0-e00c-415f-a86f-df9f68fabb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.124869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c215e-b6cb-40af-be16-5bc014accdc2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:21.132675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f35509f0-e00c-415f-a86f-df9f68fabb32 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:21.133928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f35509f0-e00c-415f-a86f-df9f68fabb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:21.200872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:21.201577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.202716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.203158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.203870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:21.204173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:21.205569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:21.222162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 172/339] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:21.227764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:21.261972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 168/340] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:21.285460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2da65-ff73-42d4-9fc4-9b5c4469aef6 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.286073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2da65-ff73-42d4-9fc4-9b5c4469aef6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.286549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2da65-ff73-42d4-9fc4-9b5c4469aef6 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.288428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 169/341] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:21.293412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057241d-275e-4d00-8ed5-6dcda98203d6 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.294262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057241d-275e-4d00-8ed5-6dcda98203d6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.294991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057241d-275e-4d00-8ed5-6dcda98203d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.317936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:21.332854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0057241d-275e-4d00-8ed5-6dcda98203d6 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:21.334068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0057241d-275e-4d00-8ed5-6dcda98203d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:21.370124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:21.416435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f3304-2223-4358-b3a1-42a1758bfe18 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:21.495100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a746e-16fc-411b-becc-0ba857672605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:21.495798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a746e-16fc-411b-becc-0ba857672605 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.496082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a746e-16fc-411b-becc-0ba857672605 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.496355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a746e-16fc-411b-becc-0ba857672605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.530697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 170/342] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:21.598761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596b45-7196-43e3-baf8-e508edb6fa18 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.599068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596b45-7196-43e3-baf8-e508edb6fa18 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.599256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596b45-7196-43e3-baf8-e508edb6fa18 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.600880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 171/343] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:21.604884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596b45-7196-43e3-baf8-e508edb6fa18 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:21.608901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a746e-16fc-411b-becc-0ba857672605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:21.610791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a746e-16fc-411b-becc-0ba857672605 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:21.659820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:21.719195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 173/344] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:31:21.720813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.721329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.721810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.741695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:21.743251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:21.817283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 174/345] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:21.818576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:21.822595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596b45-7196-43e3-baf8-e508edb6fa18 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:21.880195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:21.880762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.881043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.881315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.894750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:21.899912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:21.907236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:21.908174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:21.908959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:21.912476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 172/346] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:21.930230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:21.938906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578474cd-0f2c-4430-867b-d5d8113e1703 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:21.956995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:21.978225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:21.979320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:21.979856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:21.980450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:21.981831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:21.983696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:21.985852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 175/347] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:21 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:21.986912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.100347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aca9a-d796-47f6-bc70-62a98de87ff6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.106149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5960f-2978-4688-8885-fa9e43e30ef7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.167745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.170069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.171438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.172334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.173215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.173686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.174295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.174619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.175278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:22.175635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:22.177055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:22.190078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 173/348] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:22.190262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:22.191324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.194698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:22.207257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:22.208270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:22.209081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 176/349] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:22.209749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.310259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:22.355226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb6e81-3fdf-40a8-8786-42c170738a51 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.361957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b158ddd-965b-45ab-997a-422f225b1342 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.394333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.395349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.395876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.396335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.402838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:22.404664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.405177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:22.405662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.405993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.406270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.408922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 174/350] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:22.411045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.427755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:22.432901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:22.433794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:22.434570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:22.439659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 177/351] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:22.440516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.506823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:22.548376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27709250-5750-4792-8b72-dcf8b376ec86 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.566014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3574-dd5a-47a4-9291-ed2678ecf755 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.598185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.599299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.599807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.600338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.606528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.607145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.607419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.607746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.608419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:22.608721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:22.610096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:22.611820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:22.622823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 178/352] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:22.622823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.695710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:22.697717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:22.757023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf280c0-6673-4c20-8077-cae16c7a75ea None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.783039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:22.803897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:22.804456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:22.804731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:22.805248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:22.805982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=d98f31b12d634fe4849900dfd66e2969) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:22.806283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:22.808341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:22.815939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 179/353] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:22.820466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.843747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 175/354] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:22.844985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:22.901018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:22.985772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bc374-5919-47de-8ad6-d6f6f7ac84c1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:22.993657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301313ba-2d7d-4508-ad14-2ea87883a8cc None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.028398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.029015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.029293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.029629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.030301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:23.030605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.031998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b059f9-adf5-4798-9d72-05462d89e94c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.040807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.040980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 180/355] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.041246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.041520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.041843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66727224-eab0-4ab9-9e78-772f346e4c7a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.041942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] PATH_INFO: `/v3/users/30ead664714f42b7906f1bde1ddaf9e1` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.042127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66727224-eab0-4ab9-9e78-772f346e4c7a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.042401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66727224-eab0-4ab9-9e78-772f346e4c7a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.043175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=30ead664714f42b7906f1bde1ddaf9e1)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.051819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 181/356] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:23.051819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c1f25-7467-4059-9185-b7bac95ddec9 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.051819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c1f25-7467-4059-9185-b7bac95ddec9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.052121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.052206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c1f25-7467-4059-9185-b7bac95ddec9 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.052206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 182/358] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:23.052326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 176/357] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:31:22 2022] GET /identity/v3/users/30ead664714f42b7906f1bde1ddaf9e1 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.052402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c1f25-7467-4059-9185-b7bac95ddec9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.052521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.121869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.180305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9ff3-5aaa-4208-a3e2-99327c5b1ad8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.192468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.193512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c1f25-7467-4059-9185-b7bac95ddec9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.231218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.232250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.233263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.233820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ecc6a-f453-4efc-8a75-3838dd802079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.234227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.234652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ecc6a-f453-4efc-8a75-3838dd802079 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.234926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ecc6a-f453-4efc-8a75-3838dd802079 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.235201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ecc6a-f453-4efc-8a75-3838dd802079 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.236583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ecc6a-f453-4efc-8a75-3838dd802079 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.237972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ecc6a-f453-4efc-8a75-3838dd802079 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.239838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 183/359] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.240803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258cd8-fd5c-40da-8262-6b864163b5d0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.241082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258cd8-fd5c-40da-8262-6b864163b5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.241390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258cd8-fd5c-40da-8262-6b864163b5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.265655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-91258cd8-fd5c-40da-8262-6b864163b5d0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:23.266975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-91258cd8-fd5c-40da-8262-6b864163b5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:23.318664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.320202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.349431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 184/360] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:23.350980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.351263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.351551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.375918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:23.377550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:23.431626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 177/361] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:23.432196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.433023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.442902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.463099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 185/362] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:23.464254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.558559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d810677-6ece-4823-a970-73190e8e66aa None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.571859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f1a12-a069-43f1-acd1-72d4e1842181 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.609645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.610669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.611223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.611707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.613047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.614665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.616451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 186/363] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.617375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.624544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.625248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.625593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.625930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.626723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] RBAC: Adding query filter params (role.id=4bad7209656f49bfbc3d5d94c8824368, user.id=30ead664714f42b7906f1bde1ddaf9e1, scope.system=all) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:23.627112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.628647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.641624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 178/364] 10.222.0.55 () {56 vars in 1308 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/role_assignments?user.id=30ead664714f42b7906f1bde1ddaf9e1&scope.system=all&role.id=4bad7209656f49bfbc3d5d94c8824368 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.642529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.669923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.723706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.738737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f819e0-723d-446a-8048-f94d0f67fba6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.777236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.778244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.778787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.779297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.781661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.783506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.784777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f369356-3974-4b6d-88fe-460ce727ddd6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.785699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 187/365] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.786775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.837216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.837762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.838037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.838380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.839051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=d98f31b12d634fe4849900dfd66e2969) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:23.839355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:23.840946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:23.851612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 179/366] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:23.852709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:23.907956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6aa78-31d2-409f-a609-bee80f787717 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:23.923571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:23.957689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:23.958350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:23.958626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:23.958893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:23.973316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:23.979946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:23.981077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:23.981865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:23.987102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 188/367] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:23.988360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:24.033588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd767167-0e8f-474d-b389-451c1d5451ac None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.080792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:24.098565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.099140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.099437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.099714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.102944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:24.104473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:24.106387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 180/368] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:24.108442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:24.144876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f59336-0217-4478-a091-5e719284b2c9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.193460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.194458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.195495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.195914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.209775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:24.214823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:24.215723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 189/369] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:24.216908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:24.255334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd11e8-6ad5-4097-902c-df2d29824b20 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.258873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:24.299300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.300443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.300885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.301509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.302245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:24.302626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:24.304331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:24.317042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 181/370] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:24.317973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:24.347888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6fd19-0821-4107-9263-dee9c3949654 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.393183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.394099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.394554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.395100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.396086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:24.396632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:24.398296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:24.401398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:24.406481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 190/371] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:24 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:24.407625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:24.506982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b1dc-b81b-4bc1-a2e5-169688b4b6cc None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.567021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.567731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.568010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.568343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.570146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:24.570467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:24.571970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:24.586573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 182/372] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:24.598013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:24.620580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:24.632957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33677610-0dda-420b-85e6-ed39b75f0d1f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.695365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.696249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.696702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.697118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.749290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0823cee-8b70-4fd0-ac13-a5907c0ca93b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:24.783891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:24.791256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:24.791950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:24.792236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:24.792511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:24.863462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:24.865642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:24.919791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:24.922519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:24.991138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:25.002026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 191/373] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:25.007885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:25.127456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb02266-96f1-4d90-9244-8c2eb948c430 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:25.165682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:25.166319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.166594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.166862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.168291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:25.169728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:25.172587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 192/374] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:25 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:25.178458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:25.184949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 183/375] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:25.190959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:25.227648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:25.276015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:25.353052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fea2d1-d287-4f5f-846a-fa92f18b402f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:25.405500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408de8e-1adf-45b1-be39-49ee544a1ba2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:25.409087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:25.409725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.410003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.410272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.411393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:25.412876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:25.417618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 193/376] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:25 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:25.421330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:25.489424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:25.490069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.490344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.491395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] PATH_INFO: `/v3/projects/6d64e2847698419baef7cbb0bac87311` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.495545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] RBAC: Authorizing `identity:get_project(project_id=6d64e2847698419baef7cbb0bac87311)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:25.497301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:25.500711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 184/377] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:25 2022] GET /identity/v3/projects/6d64e2847698419baef7cbb0bac87311 => generated 344 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:25.506161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:25.558286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d56468-4f54-4ee4-b8a6-66e6bb0bbbef None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:25.619717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:25.632953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:25.633640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.633915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.634176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.634932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:25.635271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:25.636753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:25.654106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 194/378] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:25 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:25.667811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:25.684251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a101662-4f3e-471c-9c9c-411a155db746 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:25.729225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a45c86-8436-40e5-a474-c384a9044dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:25.729923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a45c86-8436-40e5-a474-c384a9044dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.730201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a45c86-8436-40e5-a474-c384a9044dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.730527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a45c86-8436-40e5-a474-c384a9044dc7 None admin] PATH_INFO: `/v3/groups/f85d85f4000245f4b2c8d5c0097d357a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.736190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a45c86-8436-40e5-a474-c384a9044dc7 None admin] RBAC: Authorizing `identity:get_group(group_id=f85d85f4000245f4b2c8d5c0097d357a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:25.737653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a45c86-8436-40e5-a474-c384a9044dc7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:25.739334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 185/379] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:25 2022] GET /identity/v3/groups/f85d85f4000245f4b2c8d5c0097d357a => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:25.745476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:25.770692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c464-7a04-4b0a-8258-f26aa064db80 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.771030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c464-7a04-4b0a-8258-f26aa064db80 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.771290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c464-7a04-4b0a-8258-f26aa064db80 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.772696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 186/380] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:25.779054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.780294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.780564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.801900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:25.803526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:25.895267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e994b8-d0ce-49a4-973d-335a5e499335 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:25.897825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 187/381] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:25.937200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:25.937866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:25.938147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:25.938472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:25.939211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] RBAC: Adding query filter params (group.id=f85d85f4000245f4b2c8d5c0097d357a, role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=6d64e2847698419baef7cbb0bac87311) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:25.939550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:25.940974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:25.947110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 195/382] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:25 2022] GET /identity/v3/role_assignments?group.id=f85d85f4000245f4b2c8d5c0097d357a&scope.project.id=6d64e2847698419baef7cbb0bac87311&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:26.204269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:26.295057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:26.317241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d4ad-734a-4582-bb97-1fdd6a833fc1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:26.362812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:26.363494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:26.363734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:26.363999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:26.375574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:26.380395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:26.381262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:26.382040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 188/383] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:26.387725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:26.454827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:26.521381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a5230-567e-4a47-a7f2-7eb4888e7245 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:26.564101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:26.564728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:26.565003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:26.565271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:26.565960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:26.566262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:26.567767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175fa0-2017-4596-8731-36871f8912ae None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:26.579912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 196/384] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:26.585631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:26.738441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b8d7-9b14-4ab3-8043-bdf74445533d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:26.760564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7aa70-ef91-4e11-b0fc-116547c4e7bd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:26.761070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7aa70-ef91-4e11-b0fc-116547c4e7bd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:26.761511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7aa70-ef91-4e11-b0fc-116547c4e7bd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:26.763068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 197/385] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:26.772317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:26.772920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:26.773388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:26.793266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:26.793904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:26.794181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:26.794449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:26.795953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:26.797424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:26.800102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:26.801580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:26.803850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 189/386] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:26 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:26.809418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:26.822059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:26.879883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 198/387] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:26.962144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234e87d-5fa4-4aea-afe5-1b2f3bfa2b41 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:27.015490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1988ac1-c292-40e7-8d87-bd1e70b75ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:27.016061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1988ac1-c292-40e7-8d87-bd1e70b75ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.016336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1988ac1-c292-40e7-8d87-bd1e70b75ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.016603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1988ac1-c292-40e7-8d87-bd1e70b75ed5 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.017745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1988ac1-c292-40e7-8d87-bd1e70b75ed5 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:27.019178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1988ac1-c292-40e7-8d87-bd1e70b75ed5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:27.020824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 190/388] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:26 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:27.026618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:27.119885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:27.183674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5b6a-368b-456c-9d9d-a791e91e44a1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:27.192092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec028-6dae-4cfa-a058-eaaa5c69a286 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.192409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec028-6dae-4cfa-a058-eaaa5c69a286 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.192636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec028-6dae-4cfa-a058-eaaa5c69a286 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.194301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 191/389] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:27.199019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.199301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.199624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.233387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:27.237083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:27.256760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:27.257959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.258434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.258895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969/groups/b00fffe312b44712baab673e22f66c51/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.263103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] RBAC: Authorizing `identity:create_grant(project_id=d98f31b12d634fe4849900dfd66e2969, group_id=b00fffe312b44712baab673e22f66c51, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:27.265553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:27.286917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 199/390] 10.222.0.55 () {58 vars in 1442 bytes} [Sat Sep 24 19:31:27 2022] PUT /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969/groups/b00fffe312b44712baab673e22f66c51/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:27.288385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:27.377902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 192/391] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:27.418766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c59a7-f87b-4e39-8291-caae568d986e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:27.434179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:27.465274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:27.466338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.467193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.467770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.480890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:27.487336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:27.488288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:27.489068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:27.491916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 200/392] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:27.500610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:27.517759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:27.629583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ec415-8144-40a8-809b-fa53fb6a43a0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:27.671046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:27.671868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.672146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.672414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.673085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:27.673388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:27.674769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:27.687133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 193/393] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:27.692690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:27.768795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:27.809779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707e305-32ed-4630-9d2c-af462317cfd4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:27.869819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:27.870457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.870733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.870999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:27.872233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:27.873726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:27.878713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 201/394] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:27 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:27.887927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:27.910073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:27.950627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e4c0-4100-45e0-8fac-4e3367ed8150 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:27.963796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:27.995267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:27.996085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:27.996367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:27.996640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.009370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:28.015877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:28.016756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:28.017533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:28.018371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 194/395] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:28.024239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:28.085572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6331ec-eb57-4e96-9f6a-360d961b0347 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:28.167300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82880612-b548-4cb7-93b8-3ce3895ed897 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:28.169770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:28.171358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.171687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.171962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.176154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:28.177612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:28.182789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 202/396] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:27 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:28.185103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:28.207843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:28.208522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.208832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.209101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.209816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] RBAC: Adding query filter params (name=member) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:28.211083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:28.211570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:28.213195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:28.222994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 195/397] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:28.228726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:28.347049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e830f-9a80-4550-bb14-0b745e46d381 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:28.388297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:28.389532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.390006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.390633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.392008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:28.393700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:28.395810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 196/398] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:28 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:28.402285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:28.473550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:28.511304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9253b51-16fc-464b-8e26-dbc0d3b6bba2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:28.522287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:28.525378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46756c4-7330-4c1e-8d37-dcb05e651b45 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:28.573334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:28.573915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.574191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.574493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.575614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:28.577527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:28.579552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 197/399] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:28 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:28.588542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:28.608767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5d2af-80e6-41cd-9287-5bc66d96989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:28.609369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d2af-80e6-41cd-9287-5bc66d96989c None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.609686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d2af-80e6-41cd-9287-5bc66d96989c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.609960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d2af-80e6-41cd-9287-5bc66d96989c None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969/groups/b00fffe312b44712baab673e22f66c51/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.616742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5d2af-80e6-41cd-9287-5bc66d96989c None admin] RBAC: Authorizing `identity:create_grant(project_id=d98f31b12d634fe4849900dfd66e2969, group_id=b00fffe312b44712baab673e22f66c51, role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:28.618220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5d2af-80e6-41cd-9287-5bc66d96989c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:28.657953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 203/400] 10.222.0.55 () {58 vars in 1442 bytes} [Sat Sep 24 19:31:28 2022] PUT /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969/groups/b00fffe312b44712baab673e22f66c51/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:31:28.738077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5fe2b-9e4b-4231-8ab6-7468c1379f0f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:28.778383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:28.790906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c5fe4-8f17-4ec8-b85c-b194a3490de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:28.792029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c5fe4-8f17-4ec8-b85c-b194a3490de3 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.792516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c5fe4-8f17-4ec8-b85c-b194a3490de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.793009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c5fe4-8f17-4ec8-b85c-b194a3490de3 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/groups/b00fffe312b44712baab673e22f66c51/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.796176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c5fe4-8f17-4ec8-b85c-b194a3490de3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a, group_id=b00fffe312b44712baab673e22f66c51, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:28.797989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c5fe4-8f17-4ec8-b85c-b194a3490de3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:28.835064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 198/401] 10.222.0.55 () {58 vars in 1442 bytes} [Sat Sep 24 19:31:28 2022] PUT /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/groups/b00fffe312b44712baab673e22f66c51/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:28.905714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eac27f-0bcd-4f36-8f17-e28ef67d6675 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.906168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eac27f-0bcd-4f36-8f17-e28ef67d6675 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.906357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eac27f-0bcd-4f36-8f17-e28ef67d6675 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.908123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 204/402] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:28.913716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66567d5-912c-4801-b05d-5c85bf3d6094 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:28.914278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66567d5-912c-4801-b05d-5c85bf3d6094 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:28.914804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66567d5-912c-4801-b05d-5c85bf3d6094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:28.936172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e66567d5-912c-4801-b05d-5c85bf3d6094 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:28.937749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e66567d5-912c-4801-b05d-5c85bf3d6094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:29.023342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 199/403] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:29.148098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6005d-08b0-4eee-9c5d-384435362946 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:29.148423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6005d-08b0-4eee-9c5d-384435362946 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:29.148653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6005d-08b0-4eee-9c5d-384435362946 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:29.150295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 205/404] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:29.158034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:29.158573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:29.159182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:29.179100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:29.180851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:29.259045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 200/405] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:29.343604 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6005d-08b0-4eee-9c5d-384435362946 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:29.452468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6005d-08b0-4eee-9c5d-384435362946 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:29.487951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:29.488956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:29.489429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:29.489959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:29.501780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:29.507277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:29.508174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:29.508963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:29.514597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 206/406] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:29.518945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:29.572629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:29.626692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477f2d5-1eda-4b42-8e9a-864c001bd5c5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:29.661972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:29.662586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:29.662861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:29.663126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:29.663803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:29.664099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:29.665467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:29.674860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 201/407] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:29.680781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:29.691356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:29.759538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:29.812291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b62b2-cb8d-48ea-82b1-039813ab4ec9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:29.824905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:29.852061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:29.853151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:29.853643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:29.854085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:29.866471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:29.872886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:29.873818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:29.874621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:29.877669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 202/408] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:29.883642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:29.939033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ccd8b-7945-4736-857c-90c7f1dd5914 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:29.998672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.000752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.001859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.002459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] PATH_INFO: `/v3/groups/f85d85f4000245f4b2c8d5c0097d357a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.005592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] RBAC: Authorizing `identity:get_group(group_id=f85d85f4000245f4b2c8d5c0097d357a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.008385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.010666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 207/409] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:29 2022] GET /identity/v3/groups/f85d85f4000245f4b2c8d5c0097d357a => generated 229 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:30.022039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.081441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db43e8-b701-470a-b795-c2c1a466d19b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.125711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.126272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.126551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.126820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.127670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:30.127906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.129280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd6f00-89b9-4690-9c76-6d2b99054483 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.142746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 203/410] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:30.142746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194819-2054-4afc-b095-8df787d5e77f None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.142746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194819-2054-4afc-b095-8df787d5e77f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.142746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194819-2054-4afc-b095-8df787d5e77f None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.142746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 204/411] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:30.148386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93194819-2054-4afc-b095-8df787d5e77f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.178775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:30.187462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:30.188679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b331cdf-1e31-40cf-a23a-7fe71227c970 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.229035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63bb76-2a7d-40f9-8bc1-4a8d66da4920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.229665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63bb76-2a7d-40f9-8bc1-4a8d66da4920 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.229942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63bb76-2a7d-40f9-8bc1-4a8d66da4920 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.230212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63bb76-2a7d-40f9-8bc1-4a8d66da4920 None admin] PATH_INFO: `/v3/projects/6d64e2847698419baef7cbb0bac87311` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.231330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63bb76-2a7d-40f9-8bc1-4a8d66da4920 None admin] RBAC: Authorizing `identity:get_project(project_id=6d64e2847698419baef7cbb0bac87311)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.234241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63bb76-2a7d-40f9-8bc1-4a8d66da4920 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.236567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 208/412] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:30 2022] GET /identity/v3/projects/6d64e2847698419baef7cbb0bac87311 => generated 344 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:30.237834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.238168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.238446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.259200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:30.260474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:30.281538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93194819-2054-4afc-b095-8df787d5e77f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.319581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.320317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.320736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.320923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.322124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.323647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.330606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 205/413] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:30 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:30.331632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.339952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 209/414] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:30.347460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.483777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:30.492988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562174d-589e-4181-86bf-b89e874b0a63 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.538986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.539640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.539865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.540098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.540328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:30.544157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.546962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.548881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 210/415] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:30 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:30.550423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3737e7f-35e7-4fae-b2cf-77db7301fb9b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.556473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.644667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.645357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.645638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.645911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] PATH_INFO: `/v3/projects/6d64e2847698419baef7cbb0bac87311/groups/f85d85f4000245f4b2c8d5c0097d357a/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.653257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] RBAC: Authorizing `identity:create_grant(project_id=6d64e2847698419baef7cbb0bac87311, group_id=f85d85f4000245f4b2c8d5c0097d357a, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.655365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.668490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a6ec-b638-49fa-9919-f2383523648c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.679064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 206/416] 10.222.0.55 () {58 vars in 1442 bytes} [Sat Sep 24 19:31:30 2022] PUT /identity/v3/projects/6d64e2847698419baef7cbb0bac87311/groups/f85d85f4000245f4b2c8d5c0097d357a/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:30.680748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.726871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.727953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.728376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.728958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/groups/b00fffe312b44712baab673e22f66c51/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.731340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a, group_id=b00fffe312b44712baab673e22f66c51, role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:30.733293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:30.766377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 211/417] 10.222.0.55 () {58 vars in 1442 bytes} [Sat Sep 24 19:31:30 2022] PUT /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a/groups/b00fffe312b44712baab673e22f66c51/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:30.852180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c7077-4fcb-4deb-a10d-fef13c24d9ee None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:30.892037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:30.892735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:30.893010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:30.893323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:30.905953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:30.911700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:30.912640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:30.913437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 207/418] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:30.919101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:30.943286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:31.031736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c240-0eae-44f2-ae4c-e6f8cc8b2bc6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:31.069529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:31.070138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.070411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.070680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.071349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:31.071740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:31.073164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d2443-454c-4051-ae96-64553035a6dd None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:31.083546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 212/419] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:31.089239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:31.113246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:31.196479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34107fb3-ab55-436c-9b60-87e307c72e77 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:31.232125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:31.232763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.233095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.233364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.328127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:31.329640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:31.359937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:31.449749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc40dd-de22-4756-9803-f95e3a29dda0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.450079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc40dd-de22-4756-9803-f95e3a29dda0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.450289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc40dd-de22-4756-9803-f95e3a29dda0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.452176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 213/420] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:31.457835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c86f2-4981-43a2-87a1-f6ad706d1eff None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.458083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 208/421] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:31.458184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c86f2-4981-43a2-87a1-f6ad706d1eff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.458387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c86f2-4981-43a2-87a1-f6ad706d1eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.463588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:31.492587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-193c86f2-4981-43a2-87a1-f6ad706d1eff None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:31.493776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-193c86f2-4981-43a2-87a1-f6ad706d1eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:31.589419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 214/422] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:31.590748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e600b-e2e4-4b98-b6e5-39741af7659e None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.591030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e600b-e2e4-4b98-b6e5-39741af7659e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.591294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e600b-e2e4-4b98-b6e5-39741af7659e None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.592166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da1bdd-498d-4c9f-91ad-1218a866e3b7 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:31.593317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 215/423] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:31.598654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.598928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.599191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.618002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:31.618325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:31.619457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:31.628796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:31.638844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:31.639955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.640484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.641032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.648200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:31.649931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:31.651921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 209/424] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:31 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:31.657917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:31.748868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 216/425] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:31.827049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e833-e81e-4fb0-ad0c-2c0f65c6940e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:31.874038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:31.874762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:31.874987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:31.875189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:31.876339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:31.877880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:31.879470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 210/426] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:31 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:31.885363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:31.927726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:31.994398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68017238-21f5-43c1-8350-ebaeda26c06e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.031761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.032295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.032678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.032950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.033680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=d98f31b12d634fe4849900dfd66e2969) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:32.033984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:32.035501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:32.040129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:32.042720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 217/427] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:31 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:32.076857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.086353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:32.155114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039987e-cd21-49b9-8fcf-bc97295502f2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.238716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdd91-5f54-45b4-b261-f97549aeb999 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.240796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.241450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.241719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.241982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.262963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:32.271284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:32.272251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:32.273118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:32.279541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 211/428] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:32.282089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.302961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.303624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.303901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.304171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.319345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:32.326091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:32.327318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 19:31:32.328136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:32.331191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 218/429] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:32.333518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.408077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533f262-e5f2-4034-a437-229948e523a1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.423280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:32.433194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:32.462157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.462770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.463044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.463312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.464008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:32.464308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:32.465697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:32.476712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bde25-f83a-43f2-b03f-6cf09a55dfc4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.484319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 212/430] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:32.486005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.518888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.519496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.519785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.520043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.520707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:32.521006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:32.522372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:32.538971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 219/431] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:32.548099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.629541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadc873-2f29-4362-9681-b1b98767ebe6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.677297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.677910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.678187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.678461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.697555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994d854-9c33-482d-89a0-41889f9edec2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:32.739549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:32.751917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:32.754986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:32.755781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:32.756346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:32.756865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:32.779034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:32.781331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:32.849668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:32.851337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:32.894376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 213/432] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:32.899962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.974129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 220/433] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:32.983528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:32.991190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:33.015117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:33.018823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090ccf0-7655-4a20-af6d-10873b9edd65 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:33.058125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:33.059072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.059750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.061225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] PATH_INFO: `/v3/projects/d98f31b12d634fe4849900dfd66e2969` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.062375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] RBAC: Authorizing `identity:get_project(project_id=d98f31b12d634fe4849900dfd66e2969)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:33.063945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:33.066674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 214/434] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/projects/d98f31b12d634fe4849900dfd66e2969 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:33.071946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:33.098324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508af3-a05a-4e7e-9080-714c0bb8c837 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:33.136819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:33.137692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.138275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.138714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.140629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:33.142207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:33.144350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 221/435] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:32 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:33.151549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:33.204594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62a06f-8c46-4d69-973c-cf3a32a41990 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:33.242194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:33.243116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.243555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.243956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.245213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:33.246801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:33.248886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 215/436] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:33 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:33.255308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:33.274578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:33.276491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:33.283599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df5b5f-814a-4b71-862a-3a2679848468 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:33.324319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:33.325044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.325679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.326062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.327249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:33.329081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:33.336352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 222/437] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:33 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:33.337482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:33.401846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f339e8-5e39-454f-b9ae-37f5d2cf56ab None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:33.453770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:33.454852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.455231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.455623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.457048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=d98f31b12d634fe4849900dfd66e2969) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:33.457762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:33.459728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b5309-8d65-4215-a302-dc6c4342c466 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:33.463683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60349a4a-b5c6-4a6b-adf5-c83a4888af20 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:33.475678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 216/438] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:33 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=d98f31b12d634fe4849900dfd66e2969&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:33.477248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771958f-3e6c-4777-80ea-879f8aff3f53 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.477537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771958f-3e6c-4777-80ea-879f8aff3f53 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.477825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771958f-3e6c-4777-80ea-879f8aff3f53 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.479625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 217/439] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:33.486093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.486367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.486639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.506862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:33.508121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:33.510307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:33.511037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.511386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.511778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.512517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=1abc6940fcf2429aab6c6fc960488040, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:33.513005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:33.515064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fae609-deeb-4c44-a70f-9bc2ffdd8e6b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:33.522823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 223/440] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:33 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=1abc6940fcf2429aab6c6fc960488040 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:33.584647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062b29e-b9a0-4677-b95c-059ffd006759 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.584960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062b29e-b9a0-4677-b95c-059ffd006759 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.585348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062b29e-b9a0-4677-b95c-059ffd006759 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.587147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 224/441] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:33.596710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:33.596710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:33.596710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:33.603065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 218/442] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:33.614763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:33.616113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:33.647903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:33.697721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 225/443] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:33.887044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:33.898520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:33.972926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:33.996786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148eca91-d711-4280-bcab-d376da4faf43 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.040697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.041453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.041731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.041999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.058789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:34.064799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:34.065670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:34.066458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 219/444] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:34.072122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.102329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8e72e-3780-475e-8734-33bc14a4a4b1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.140876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.141605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.141876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.142141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.148444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.154395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:34.159556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:34.160479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:34.161276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 226/445] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:34.166550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.186426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60b4cb-921f-403d-8bfd-d81d049bb6a0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.186905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.220178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.220783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.221065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.221337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.221995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:34.222300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.223773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.231780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 220/446] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:34.237890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.267817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c5447-97cb-49a3-8a3a-62e0d128d5d2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.301125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.301722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.301997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.302264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.302911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:34.303206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.304608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.313131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 227/447] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:34.318715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.338570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a19bdd-2e3b-4691-96eb-8d8c0a0f8f59 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.366417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.372940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.373982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.374256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.374521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.397352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.417186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806149b8-1e7c-417b-bb75-f5975c8060c2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.441955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.443413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.451912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.452522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.452805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.453071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.527766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.529178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.543664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 221/448] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:31:34.549999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.587274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.613489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.633232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 228/449] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:31:34.638952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.656896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bcfca-b5d3-492e-9a47-4faf077c654b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.692859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.693510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.693787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.694058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] PATH_INFO: `/v3/projects/6d64e2847698419baef7cbb0bac87311` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.695180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d64e2847698419baef7cbb0bac87311)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.696681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.698301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 222/450] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/projects/6d64e2847698419baef7cbb0bac87311 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:34.704000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.742810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01f6e9-4052-4ec3-b964-f10e46528152 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.775375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.776025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.776302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.776571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] PATH_INFO: `/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.777680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] RBAC: Authorizing `identity:get_project(project_id=b683c9ec1f9f4f77af7a2e64bd4b643a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.779999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.783467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 229/451] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/projects/b683c9ec1f9f4f77af7a2e64bd4b643a => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:34.787525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.806884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.811364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1247e3-79c4-4e53-b276-d19634923ef9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.838677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:34.845823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.846455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.846736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.847007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] PATH_INFO: `/v3/groups/f85d85f4000245f4b2c8d5c0097d357a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.848158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] RBAC: Authorizing `identity:get_group(group_id=f85d85f4000245f4b2c8d5c0097d357a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.849620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.851194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 223/452] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/groups/f85d85f4000245f4b2c8d5c0097d357a => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:34.857044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.890371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff21c62-95b5-4572-aea6-9834e8558213 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.925066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.925687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.925962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.926254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] PATH_INFO: `/v3/groups/b00fffe312b44712baab673e22f66c51` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.927347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] RBAC: Authorizing `identity:get_group(group_id=b00fffe312b44712baab673e22f66c51)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.928824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.930797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 230/453] 10.222.0.55 () {56 vars in 1180 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/groups/b00fffe312b44712baab673e22f66c51 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:34.936537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:34.956420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae6d6d-27b9-4329-bcef-ed40543b3700 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:34.990100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:34.990702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:34.990974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:34.991243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:34.991994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] RBAC: Adding query filter params (group.id=f85d85f4000245f4b2c8d5c0097d357a, role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=6d64e2847698419baef7cbb0bac87311) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:34.992295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:34.993712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534814e-2e0d-425c-9ba0-dae7ebe0b3b6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:34.999897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 224/454] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/role_assignments?group.id=f85d85f4000245f4b2c8d5c0097d357a&scope.project.id=6d64e2847698419baef7cbb0bac87311&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:35.040343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143310-3aff-4ad6-9d30-5757f316196d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:35.062521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:35.082944 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:35.083595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:35.083875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:35.084142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:35.084876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] RBAC: Adding query filter params (group.id=b00fffe312b44712baab673e22f66c51, role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:35.085183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:35.086621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d07843-8188-4a50-befd-416718e1b95e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:35.093799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 231/455] 10.222.0.55 () {56 vars in 1376 bytes} [Sat Sep 24 19:31:34 2022] GET /identity/v3/role_assignments?group.id=b00fffe312b44712baab673e22f66c51&scope.project.id=b683c9ec1f9f4f77af7a2e64bd4b643a&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:37.947053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75afeb-14c6-463c-87da-0053a5e996c6 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:37.948005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75afeb-14c6-463c-87da-0053a5e996c6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:37.948427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75afeb-14c6-463c-87da-0053a5e996c6 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:37.950340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 225/456] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:37.957326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:37.958132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:37.958551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:37.984523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:37.985982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:38.050701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50813579-5779-4a1a-8d7b-4a966a214577 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.051337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50813579-5779-4a1a-8d7b-4a966a214577 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.051694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50813579-5779-4a1a-8d7b-4a966a214577 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.053484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 226/457] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:38.059320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239a3a-5d6d-4a74-ac02-e12627779450 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.061617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239a3a-5d6d-4a74-ac02-e12627779450 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.062088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239a3a-5d6d-4a74-ac02-e12627779450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.095257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 232/458] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:38.096511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b1239a3a-5d6d-4a74-ac02-e12627779450 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:38.097939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b1239a3a-5d6d-4a74-ac02-e12627779450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:38.139830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:38.204866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 227/459] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:38.531258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:38.549601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b56993-3dd8-4874-81e9-96568301847a None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.550200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b56993-3dd8-4874-81e9-96568301847a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.550690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b56993-3dd8-4874-81e9-96568301847a None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.552474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 228/460] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:38.557946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d259-aaaa-479d-a51a-cd2e637d4989 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.558464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d259-aaaa-479d-a51a-cd2e637d4989 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.558927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d259-aaaa-479d-a51a-cd2e637d4989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.583456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-72b9d259-aaaa-479d-a51a-cd2e637d4989 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:38.583456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-72b9d259-aaaa-479d-a51a-cd2e637d4989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:38.668490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 229/461] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 19:31:38.670362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda5b9-35a8-48c0-a533-9a1638fe33fe None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.670646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda5b9-35a8-48c0-a533-9a1638fe33fe None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.670768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed6049-5d6d-4137-9442-e2cf89cfd075 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:38.670939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda5b9-35a8-48c0-a533-9a1638fe33fe None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.672298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 230/462] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:38.676888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda5b9-35a8-48c0-a533-9a1638fe33fe None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:38.684463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:38.711414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:38.712490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.713013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.713482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.724507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:38.726563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d9ecbfc2-33a9-484f-9652-6b0b5359313d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:38.744659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:38.745608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:38.746451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:38.746451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:38.746451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:38.749491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 233/463] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:38.751005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.751530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.752001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.770926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:38.772457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:38.804094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:38.820362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda5b9-35a8-48c0-a533-9a1638fe33fe None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:38.850691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 234/464] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:38.852104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:38.864733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:38.865754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:38.866259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:38.866663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:38.878664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:38.880569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:38.898047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:38.899026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:38.900011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:38.900011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:38.900011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:38.901882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 231/465] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:38.908091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:38.954905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:38.971849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d3fe3-6152-4f85-83f6-59f873c1b9c0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.020747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.022091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.022779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.023192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.024254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:39.024721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.026884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.034192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2ef4b-f01a-49ee-a2c6-754496477440 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.042664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 235/466] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:39.043628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.082012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.083284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.083724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.084378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.085195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:39.085849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.088116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.113369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 232/467] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:39.114395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.117212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:39.190089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8e8dc-80ae-4279-a26b-e0f9916f934a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.231666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.232453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.233278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.233278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.239097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba6bb1-28d0-48aa-adce-d0b7b1be6cfc None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.244101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.245575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:39.260629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:39.261545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:39.262385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:39.262385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:39.262385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:39.262385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 236/468] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:39.262592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.267512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:39.283844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.284544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.284827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.285099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.285747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.287321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.312519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 233/469] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:39 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:39.313408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.394148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab80e57-e997-4988-9cad-40acbba4dce1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.411799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:39.441961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.443087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.443677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.444131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.445016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.446853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.468355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2654b0-ee22-4150-983a-391ee69fc62b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.476272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 237/470] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:39 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:39.477705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.537408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.538409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.539073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.539576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.556409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.558402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.559174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:39.581817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:39.582820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:39.583706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:39.583706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:39.583706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:39.585860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 234/471] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:39.593610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.619669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba73e-1984-4867-8ac2-de11cf4cb6b2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.663086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.664076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.664510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.664792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.665458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:39.665764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.667180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336808-4616-462f-8569-445901fcdb4a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.683697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 238/472] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:39.689034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18f74-236a-47bb-b3e6-19d1b736a594 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.689034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18f74-236a-47bb-b3e6-19d1b736a594 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.689034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18f74-236a-47bb-b3e6-19d1b736a594 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.689034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 239/473] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:39.692519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18f74-236a-47bb-b3e6-19d1b736a594 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.730564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35ff3-75c5-4d64-b66e-df89ef5bb1f1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.743893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:39.775647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.776268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.776550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.776849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.778136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:39.778136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.779311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1432-f695-49d0-8808-1cef5cf830d4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.793341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 235/474] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:39.796563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.797557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.797557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.817324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:39.818741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:39.825663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18f74-236a-47bb-b3e6-19d1b736a594 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:39.849391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:39.867077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:39.867664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:39.867939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:39.868201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:39.868844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:39.870379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:39.899625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 240/475] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:39 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:31:39.900448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:39.903160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 236/476] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:40.031325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c003e2-1f4e-41a8-b8d4-103329beec1d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:40.073562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:40.074444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:40.074870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:40.075280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:40.076258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:40.078195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:40.113928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 241/477] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:39 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:40.253973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:40.377136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:40.380958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5806c-477b-4c96-8b6f-23616c241117 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:40.423906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:40.424759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:40.425174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:40.425577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:40.426412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:40.427956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:40.438137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.sql.core [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 19:31:40.438137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 19:31:40.438137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [parameters: {'id': '775b0cecee354ed68fc53a1a0027494a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 19:31:40.438137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81801) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application conn.query(q) Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result.read() Sep 24 19:31:40.455482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return ref.to_dict() Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 19:31:40.456571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application next(self.gen) Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application yield resource Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application next(self.gen) Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise self.value Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application session.commit() Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 19:31:40.457690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.session.flush() Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application compat.raise_( Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 19:31:40.458689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 19:31:40.459609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application conn.query(q) Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result.read() Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 19:31:40.460406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [parameters: {'id': '775b0cecee354ed68fc53a1a0027494a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:40.461183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 19:31:40.462004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:40.464344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 237/478] 10.222.0.55 () {60 vars in 1131 bytes} [Sat Sep 24 19:31:40 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:40.470579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:40.513697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:40.591540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f7aec-aa28-4f2f-a38f-1621acfc335a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:40.632890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09c31c-c178-4059-97de-00e63d4ab551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:40.633465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c31c-c178-4059-97de-00e63d4ab551 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:40.633743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c31c-c178-4059-97de-00e63d4ab551 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:40.634013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c31c-c178-4059-97de-00e63d4ab551 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:40.647619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09c31c-c178-4059-97de-00e63d4ab551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-da09c31c-c178-4059-97de-00e63d4ab551 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:40.653029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:40.654332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 19:31:40.655502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:40.658618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 242/479] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:31:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:40.660748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:40.791865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360a0a-edb8-4c83-b77e-a3368eedf058 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:40.840493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:40.841106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:40.841779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:40.842096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:40.842817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:40.843126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:40.844577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e744-85ab-472e-85ed-4dcf0cb487df None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:40.856946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 238/480] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:41.950119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8cdce-e628-45d4-bdbe-702dacaffe78 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:41.950799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8cdce-e628-45d4-bdbe-702dacaffe78 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:41.950799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8cdce-e628-45d4-bdbe-702dacaffe78 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:41.952838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 243/481] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:41.957859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b706-b641-4b7c-8d31-94b139967408 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:41.958281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b706-b641-4b7c-8d31-94b139967408 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:41.958457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b706-b641-4b7c-8d31-94b139967408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:41.981351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-73c3b706-b641-4b7c-8d31-94b139967408 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:41.983450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-73c3b706-b641-4b7c-8d31-94b139967408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:42.010933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:42.084757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 239/482] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:42.335336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b17771-9e29-4db7-80a5-26cfcf42b7ee None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.335702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b17771-9e29-4db7-80a5-26cfcf42b7ee None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.335810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b17771-9e29-4db7-80a5-26cfcf42b7ee None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.337553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 244/483] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:42.344388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.344679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.344957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.375536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:42.376814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:42.465616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 240/484] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:42.493254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b17771-9e29-4db7-80a5-26cfcf42b7ee None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:42.647282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:42.663315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b17771-9e29-4db7-80a5-26cfcf42b7ee None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:42.714855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160467c-fa07-43b7-9553-e81e34aad90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:42.715857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160467c-fa07-43b7-9553-e81e34aad90f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.716382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160467c-fa07-43b7-9553-e81e34aad90f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.716847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160467c-fa07-43b7-9553-e81e34aad90f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.738708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160467c-fa07-43b7-9553-e81e34aad90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2160467c-fa07-43b7-9553-e81e34aad90f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:42.743902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:42.744834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:42.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:42.749217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 245/485] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:42.756528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:42.779146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ccef8-b96c-4ac7-9b34-cb2a06276528 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.779785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ccef8-b96c-4ac7-9b34-cb2a06276528 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.780259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ccef8-b96c-4ac7-9b34-cb2a06276528 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.783320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 246/486] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:42.791716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.792256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.792719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.822886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:42.824779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:42.885667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:42.904438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1508a5-d8bb-479f-8960-cc8847c11d26 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:42.922726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 247/487] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:42.924270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:42.955611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:42.956232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.956514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.956788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.957461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:42.957768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:42.959143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc96c-1196-48ab-87f1-9120cf48e84d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:42.971211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 241/488] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:42.973155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0e2d7-2823-472c-a73d-31dd8d074cfd None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:42.973455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0e2d7-2823-472c-a73d-31dd8d074cfd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:42.973729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0e2d7-2823-472c-a73d-31dd8d074cfd None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:42.979073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 242/489] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:42.980721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0e2d7-2823-472c-a73d-31dd8d074cfd None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.049602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.076411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57ee75-b0b0-4ca4-94b4-3edae6f2d24c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.118611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0e2d7-2823-472c-a73d-31dd8d074cfd None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.122084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fbe39-72dd-4ebf-a22a-03087328b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.123008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fbe39-72dd-4ebf-a22a-03087328b1bf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.123466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fbe39-72dd-4ebf-a22a-03087328b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.123890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fbe39-72dd-4ebf-a22a-03087328b1bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.137195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fbe39-72dd-4ebf-a22a-03087328b1bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a15fbe39-72dd-4ebf-a22a-03087328b1bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:43.144427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:43.145474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:43.146455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:43.148673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 248/490] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:43.151336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.152032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.152875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.162871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.163594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.163977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.164349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.175754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:43.177423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.177541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:43.179119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.190076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:43.198860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:43.199977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:43.201533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:43.201533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:43.201533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:43.203630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 243/491] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:43.204891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.261004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 249/492] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:43.262067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.321185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255e6cb-bdb2-4b4e-9136-02a0d756cce1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.353879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.366035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.366554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.366808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.367089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.367829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:43.368063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.369722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.379720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 244/493] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:43.380547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.393847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e982a7-93f7-455a-944c-3344c2d89328 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.439262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.439830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.440103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.440367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.441114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:43.441414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.442941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.458261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 250/494] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:43.459337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.461904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.505924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462db9-4fde-4fc0-b750-9b7fe3a14412 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.542042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.542346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.542628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.542957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.555034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:43.562225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:43.563218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:43.564063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 245/495] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:43.564721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.583880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe57282-ec21-4f04-9409-e32678cd265c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.615072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.621096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.621722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.621999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.622264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.633101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.634532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:43.649498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:43.650411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:43.651243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:43.651243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:43.651243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:43.651243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 251/496] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:43.651243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.678207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b6bd9-caea-4788-8199-453c63c4ccf8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.702103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.715207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.715804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.716086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.716905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.725497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.728247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:43.745588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:43.746592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:31:43.747645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:43.751896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 246/497] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:43.752761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.767378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196642-9ece-44d2-9ae0-a433ff24a09d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.811410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.812490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.812783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.813058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.824081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:43.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:43.830348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:43.831127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 252/498] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:43.832482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.866537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b267-87a7-4672-916f-697b7c9dc7e3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.890408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.901290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.901896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.902182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.902452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.903110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:43.903412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.906446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.914932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 247/499] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:43.915665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:43.945598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e746f0-2184-4144-b9c0-7c40a8b5aaa3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:43.963877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:43.979725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:43.980489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:43.980773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:43.981359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:43.982089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:43.982668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:43.984115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:43.995751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 253/500] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:43.996546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.018643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a3b01-8ca1-48ac-9570-afc798b3b65b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.051187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.051833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.052108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.052435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.053157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:44.053923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.055656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75306c90-e87b-4127-b4c1-225c14f21239 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.094142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 248/501] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:44.095377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4618d2f-3102-47e4-ae56-6ffb10903f47 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.095684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4618d2f-3102-47e4-ae56-6ffb10903f47 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.095953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4618d2f-3102-47e4-ae56-6ffb10903f47 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.097293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 249/502] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:44.099767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4618d2f-3102-47e4-ae56-6ffb10903f47 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.103222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c161e-34d0-49cb-bb5f-b59d85feed7c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.105524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.136949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.137538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.137810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.138620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.139374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:44.139705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.141062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.149086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 254/503] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:44.149851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.201879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4618d2f-3102-47e4-ae56-6ffb10903f47 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.203532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.237992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.238540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.238820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.239091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.249212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.250684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.256492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190faad2-240a-4fc1-9e26-2ffe249dfe68 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:44.267131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:44.268136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:44.268970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:44.268970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:44.268970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:44.269406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 250/504] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:44.270754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.295880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.296504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.296785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.297059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.304566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.306252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bf6acdf9-565f-46b9-8af5-b7a9b2c3684f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:44.319180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:44.320246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 255/505] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:44.321497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.322252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.322252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.340392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:44.341614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:44.361034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.374645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79b99-6a6f-40d2-8870-b82f0b175957 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.407406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.408030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.408303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 256/506] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:44.408434 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.408564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.409123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.440358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.490761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.492188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.519387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390fa94-de54-4b26-b8b4-6342d50c8a34 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.553994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.555007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.555276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.555579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.566023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.567486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:44.582379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:44.583462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:44.584501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:44.584501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:44.584501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:44.584501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 257/507] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:44.584720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.598836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.603888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 251/508] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:44.603888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.659918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.704926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428b66-4283-48d1-a646-fa33819dd89e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.706584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51764c2b-8ede-4ab3-9abe-6e1393121361 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.739351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.740006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.740362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.740632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.741422 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:44.741673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.741954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.742554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.742832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.743099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.743187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.743769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:44.744067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.745455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.758303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 258/509] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:44.758941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.760204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 252/510] 10.222.0.55 () {56 vars in 1137 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:44.760951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.831534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.865200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba95e0-5785-4c88-abba-5ece54c19970 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.871868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413f64-dc98-4cc4-9441-f94198f62737 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:44.884949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:44.900108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.900711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.900990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.901268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.901903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:44.902205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.903628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.908535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:44.909154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:44.909428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:44.909699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:44.915523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 259/511] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:44.916312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:44.919932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:44.921579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:44.936528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:44.937650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:44.938505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:44.938505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:44.938505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:44.938505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 253/512] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:44.938505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.021618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf474b5c-807b-4ad4-94be-620df338b317 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.036869 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398fb51-5d9f-4915-aba7-054707be5bbc None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.047661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.056019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.056970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.057245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.057516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.069165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.070597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.072709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.073334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.073609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.073878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.082187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.083657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:45.085317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:45.086375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:45.087559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:45.087559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:45.087559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:45.089628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 260/513] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:45.090451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:45.094949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:45.095855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:45.096756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 254/514] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:45.097308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.124012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.200069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa858d15-e435-4bfa-9532-6e79b530b11f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.208092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a6a84-2ab0-4fb3-8dbd-bdaccec29d7f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.233234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.233863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.234147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.234419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.241717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.241861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.242308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.242579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.242842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.243167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:45.254778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:45.255728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 255/515] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:45.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.295145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.316773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.318300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.340622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.357784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60aa3d-6265-4843-9793-eac8488506eb None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.391214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.391844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.392121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.392390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.393017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:45.393316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.394678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.406667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 256/516] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:45.407480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.419274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 261/517] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:45.420993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.502104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.511051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9538d6-8c8b-4a1c-a5b5-7faa5755ad8b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.522817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aa54f-80ce-4186-8060-0f809f88905c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.545507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.546043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.546319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.546593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.547229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:45.547561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.548929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.558108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.558699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.558970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.559237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.560020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:45.560225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 257/518] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:45.560619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.561006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.562029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.564345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.600959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 262/519] 10.222.0.55 () {56 vars in 1139 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:45.602326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.661374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b157183-9259-47b3-8f2f-1adc86b550de None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.694051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.694655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.694935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.695210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.695964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:45.696274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.697699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.702656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20201213-3507-470a-8db4-043f8f879d6a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.716931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 258/520] 10.222.0.55 () {56 vars in 1137 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:45.717961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.741470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.742185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.742529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.742869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.750041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.751530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.753218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:45.764980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:45.765995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:45.766930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 263/521] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:45.767408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.785873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:45.825160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b07783-c476-4abc-9cfe-9fc211a7a9af None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.859474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.860102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.860386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.860659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.864408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abce9-f42d-46cc-9740-572a655b795f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:45.870527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.872020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:45.887495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:45.888616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:45.889492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:45.889492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:45.889492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:45.889492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 259/522] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:45.889492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.902242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:45.902849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:45.903126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:45.903393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:45.904982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:45.906508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:45.927802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 264/523] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:31:45 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:31:45.928611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:45.994594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.010523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937374aa-2cc1-4f08-88da-f3ba7fd87e93 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.027325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.048648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f158d84-6c2e-4b43-9c9f-048a1d242327 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.050987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.052571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.052854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.053122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.088028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.088557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.088846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.089099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.145470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.146938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.173007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.174467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.232775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.258630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 260/524] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:46.259777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.276487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.289810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 265/525] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:46.291011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.374095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88cffe-e6a7-4842-ad10-197d29b915dd None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.399262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478336-0c33-46c6-9210-cd95d629db13 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.411827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.412740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.413166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.413756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.414709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:46.415158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.416884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.434466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.435194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.435742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.436082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.436734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:46.437034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.437717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 261/526] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:46.438442 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.438625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.456429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 266/527] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:46.457263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.476206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.527995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.555215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b98f2-7c18-480b-9971-6f180429fc50 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.566021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43970db-5448-42b4-8c6d-c3ed44c8978b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.592650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.593366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.593754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.594028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.601403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.602034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.602317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.602583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.604519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.606284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.613474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.614918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:46.622978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:46.623942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:46.625060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:46.625060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:46.625060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:46.626441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 262/528] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:46.627360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:46.630633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:46.631673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:46.632874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:46.632874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:46.632874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:46.632874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 267/529] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:46.632874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.706389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.745657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a575f4-c072-47e2-b1a9-55ee0cef033e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.747160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac04c0-6fc2-4e92-b85a-54007f6c801b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.772579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.781906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.782699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.782978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.783244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.783996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=57f68f991883493dbf78c0bcfd827c1d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:46.784299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.785840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.786101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.786729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.787009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.787281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.794066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.795980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.803455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 263/530] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/role_assignments?user.id=57f68f991883493dbf78c0bcfd827c1d&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:46.803455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:46.808272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:46.809185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:46.810726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 268/531] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:46.811493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:46.932340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427bf03-73ed-4167-8c22-dfbb4711baf9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.949251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c897a-2633-4569-8378-e31cdea63cf2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:46.969974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:46.986163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.987275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.987587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.987862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.988532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:46.988839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:46.990484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:46.995376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:46.996310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:46.996777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:46.997217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:46.998078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:46.998602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.000280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.005021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 269/532] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.005932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.016475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 264/533] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.017679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.072749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:47.129258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520ed2-a1c6-43fa-9f24-47d68900d8e9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.141161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba01d0-133a-4356-860e-54c47eb38bf3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.167689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.168769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.169290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.169830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.182805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.183391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.184184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.184620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.184729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.185141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.185967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:47.186408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.187987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:47.201787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:47.202702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:47.203570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:47.203979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 265/534] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:47.204788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.210701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 270/535] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.211891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.265961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:47.329046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9b82e-6f4c-407e-a89d-bf2dee9c00ff None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.350056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927c7e-8dbb-4e09-9c3a-88f2bf6db3f1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.364404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:47.365575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.366365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.366648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.366917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.367798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:47.368172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.370256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.389368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 271/536] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.390218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.392095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.392949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.393301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.393642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.401736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.403530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:47.416731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:47.417598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:47.418435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 266/537] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:47.419800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.510092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3be04-a8ae-4b5b-ad59-4da87247fee3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.519320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:47.532970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd9e6-d5ac-4a44-8708-9f23a5fa3a3d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.552258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.552962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.553356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.553737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.554535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=6e9ce9098e884f4e9119aeccdbafd4ae) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:47.555649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.557642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.565563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 272/538] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/role_assignments?user.id=6e9ce9098e884f4e9119aeccdbafd4ae&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.566473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.570985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.571747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.572101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.572446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.573266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=b93ea821819d462097b1e76ce1ffae3f) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:47.573707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.575342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.583805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 267/539] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/role_assignments?user.id=b93ea821819d462097b1e76ce1ffae3f&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.685987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d5179-1064-4a04-b5d2-5c1fce284883 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.725397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.726025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.726306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.726577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.727294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:47.727634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.729162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b4b25-29e7-431e-90fd-6247b5cace82 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.744015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 273/540] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:47.750290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:47.814436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:47.913452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57909c0-b7ac-4fbf-aa43-59119839d80e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:47.958588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:47.959672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:47.960183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:47.960671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:47.961601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=8702de7dc5fd4fdaa05449938d461de3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:47.962104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:47.963873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee478f-5150-4b1f-b6c0-aeb99e41d387 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:47.974050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 268/541] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:47 2022] GET /identity/v3/role_assignments?user.id=8702de7dc5fd4fdaa05449938d461de3&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:48.241601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a545ca-6c6a-49b7-88b5-d4d8729553cb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:48.242235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a545ca-6c6a-49b7-88b5-d4d8729553cb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:48.242702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a545ca-6c6a-49b7-88b5-d4d8729553cb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:48.244555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 274/542] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:48.250870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:48.251775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:48.252250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:48.272642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:48.274165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:48.356225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 269/543] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:48.733127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a545ca-6c6a-49b7-88b5-d4d8729553cb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:48.772079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:48.871933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a545ca-6c6a-49b7-88b5-d4d8729553cb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:48.921399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:48.922045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:48.922318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:48.922582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:48.933808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:48.943056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:48.943983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:48.944785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 275/544] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:48.950290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:48.985728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:49.066401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e58af-5a4a-4472-897d-716b7f568d13 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:49.114058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:49.114676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:49.114951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:49.115220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:49.115902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:49.116205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:49.117581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:49.127300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 270/545] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:49.133045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:49.270151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:49.280310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cc8fd-27aa-494e-9b0d-143ee56c96e7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:49.315805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:49.316553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:49.316926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:49.317285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:49.328109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:49.329668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:49.343831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:49.344694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:49.345534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:49.345534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:49.345534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:49.345534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 276/546] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:49.350417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:49.512449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:49.525651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4cc6e-caab-479e-9dd7-eb409de45502 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:49.574127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:49.574733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:49.575011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:49.575279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:49.576038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:49.576421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:49.577810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07001a-2749-4f2d-ba9a-7867e7da1dab None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:49.598783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 271/547] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:49.604338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:49.667052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc1f96-d984-4df2-94bd-cdd1375a18c4 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:49.667394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc1f96-d984-4df2-94bd-cdd1375a18c4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:49.667620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc1f96-d984-4df2-94bd-cdd1375a18c4 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:49.669094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 272/548] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:49.675955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:49.676237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:49.676504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:49.694856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:49.696130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:49.771144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 273/549] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:49.785653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47358827-ba35-4e9f-acb1-7de1960a438a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:49.844906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:49.845536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:49.845807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:49.846076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:49.864500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:49.865948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:49.880820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:49.882529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:49.883785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 277/550] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:49.887594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:49.994180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.016823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5fcd-0fcc-4cef-90af-35b2b99d45a8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.051852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.052376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.052649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.052970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.053691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:50.053998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:50.056394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.057608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:50.060449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.077685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 274/551] 10.222.0.55 () {56 vars in 1135 bytes} [Sat Sep 24 19:31:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:50.082178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.231132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce0c5-5062-4d75-a4be-3d8dd3a28639 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.263802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e190a-48ca-463f-88be-fb0dc1f5e9f0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.290290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e55ba9-6919-4e8b-ae2f-6d3a90c6546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.290907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e55ba9-6919-4e8b-ae2f-6d3a90c6546f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.291186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e55ba9-6919-4e8b-ae2f-6d3a90c6546f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.291489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e55ba9-6919-4e8b-ae2f-6d3a90c6546f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.301993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.310546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df208de-8e5e-4c6d-9613-e50d2bfba365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.311702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df208de-8e5e-4c6d-9613-e50d2bfba365 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.311975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df208de-8e5e-4c6d-9613-e50d2bfba365 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.312252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df208de-8e5e-4c6d-9613-e50d2bfba365 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.325964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df208de-8e5e-4c6d-9613-e50d2bfba365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2df208de-8e5e-4c6d-9613-e50d2bfba365 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:50.332966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:50.333980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:50.334768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 278/552] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:50.335462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bbe6a-b481-4b4d-9b34-6f8ba1484cff None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.335547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bbe6a-b481-4b4d-9b34-6f8ba1484cff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.336358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bbe6a-b481-4b4d-9b34-6f8ba1484cff None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.341932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 279/553] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:50.342913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.343198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.343494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.363370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:50.364957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:50.389768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e55ba9-6919-4e8b-ae2f-6d3a90c6546f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:50.391561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e55ba9-6919-4e8b-ae2f-6d3a90c6546f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:50.433932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.450233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 280/554] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:50.455639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.551477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 275/555] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:50.554036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3862fd-8077-4e80-b5ec-c6437b135cdf None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.554515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3862fd-8077-4e80-b5ec-c6437b135cdf None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.555265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3862fd-8077-4e80-b5ec-c6437b135cdf None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.557239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 276/556] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:50.558128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3862fd-8077-4e80-b5ec-c6437b135cdf None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.602377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d9ed3-e900-4426-9255-4d4c25ad6f4e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.603484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.649098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.650113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.650589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.651084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.652594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:50.653138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:50.656080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900b4fe-b042-45df-b6c3-2b58074fc442 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:50.667029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3862fd-8077-4e80-b5ec-c6437b135cdf None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.675761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 281/557] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:50.675761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.675761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.675761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.699907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:50.702899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:50.719525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.720039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.720317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.720583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.731164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:50.733184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:50.748548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:50.749498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:50.750358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:50.750358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:50.750358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:50.750358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 277/558] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:50.751091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.768994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.789418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 282/559] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:50.790257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.861026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:50.874690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194a891-1a15-452c-b17d-ca1a7633049f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.897374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a998d3-4f16-412c-8447-d2631bc584c7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:50.913192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.913819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.914099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.914369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.925882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:50.927329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:50.933269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:50.933866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.934147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.934407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.935059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:50.935351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:50.936713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcda47-ac2d-4957-bcdc-5dcf11f1a2f6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:50.942582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:50.943545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:50.944454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:50.944454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:50.944454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:50.944454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 278/560] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:50.944454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:50.953345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 283/561] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:50.953345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a287e8-b82f-43e3-87ba-1c80f0e3eb87 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:50.953345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a287e8-b82f-43e3-87ba-1c80f0e3eb87 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:50.953345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a287e8-b82f-43e3-87ba-1c80f0e3eb87 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:50.953345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 284/562] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:50.953345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a287e8-b82f-43e3-87ba-1c80f0e3eb87 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.009175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.057365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f5246-b310-48ff-988f-f196ef0e8dc3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.075059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a287e8-b82f-43e3-87ba-1c80f0e3eb87 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.093293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.093923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.094207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.094483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.104457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:51.108935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.109539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:51.109730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:51.110787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:51.111846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:51.112496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.112496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.112496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:51.112496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.112748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 279/563] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:51.112748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.112949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690208-31d3-4ecb-9bec-4127faea85ce None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.118016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.126368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 285/564] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:51.127600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.127886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.128151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.145898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:51.147065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:51.212841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 286/565] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:51.213656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.222260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521e76b-785d-4bdd-9f94-9a23ceede662 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.251341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.270553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.271184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.271492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.271767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.279292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.280828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:51.292318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:51.293208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 280/566] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:51.299748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.331984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f848f5-2cbe-47c8-93be-652030a579cc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.363461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.367623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.368324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.368512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.368776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.381790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:51.386538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:51.387478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:51.389275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:51.392975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 287/567] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:51.393718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.410206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae2d3c-f907-4e91-a759-12bf25305f3d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.444620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.445225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.445622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.445896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.446551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:51.446854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.448264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.460652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 281/568] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:51.460652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.496230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.504247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac901b3e-aa1d-4e77-9f2f-37b9474fc401 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.536676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.537304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.537583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.537856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.545798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.547231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.557607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f52a00-6def-4987-ba29-f93539b12b04 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:51.558377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:51.559367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 288/569] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:51.561405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.584146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.595250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f91212-0862-4536-842b-807608e07606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.595884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f91212-0862-4536-842b-807608e07606 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.596161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f91212-0862-4536-842b-807608e07606 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.596424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f91212-0862-4536-842b-807608e07606 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.597107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f91212-0862-4536-842b-807608e07606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:51.597407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f91212-0862-4536-842b-807608e07606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.598825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f91212-0862-4536-842b-807608e07606 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.611244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 282/570] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:51.612053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f91212-0862-4536-842b-807608e07606 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.667844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c034-53d3-416b-8906-5a306b4aa274 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.703503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.704106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.704381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.704652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.705307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:51.705677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.707041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.709494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f91212-0862-4536-842b-807608e07606 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.715540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 289/571] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:51.716611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.736269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.744033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.744671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.744948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.745217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.755732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.757188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:51.771763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:51.772691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:51.773590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:51.773590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:51.773590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:51.773590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 283/572] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:51.773590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.808983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.822063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af4bb-9a33-4473-9733-867ffcf09c72 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.855393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.856059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.856334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.857151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.867050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:51.872383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:51.873362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:51.874189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:51.874820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7ba2-3c08-4265-a49e-42f77a34676b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:51.874905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 290/573] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:51.874905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.907689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:51.908284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:51.908559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:51.908825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:51.909537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:51.909835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:51.911249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:51.928548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 284/574] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:51.929309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:51.954990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:51.979115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634156-41f9-4583-ae5e-fa599386ae30 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.011366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.011984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.012258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.012521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.013287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=c72b7d859e954054848d405afa0b466f) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.013617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.015040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.019730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.020497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 291/575] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/role_assignments?user.id=c72b7d859e954054848d405afa0b466f&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.021171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.027708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157770c-048a-40a6-a00f-8b815da893ab None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.079106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.080364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.080646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.080918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.092591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.094062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:52.110679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:52.111904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:52.112952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:52.112952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:52.112952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:52.112952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 285/576] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:52.117455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.139267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fe87e-7e0a-48ea-987c-3426422e1aa8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.178157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.179136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.179345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.179901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.180547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.180939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.182229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.193643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.198391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 292/577] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.199441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.236050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022964-fe5a-45aa-9fa6-46802cc2d5d0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.274004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.274717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.274996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.275265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.275958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.276256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.277625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.291140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 286/578] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.291140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.300851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.311560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5244be-2fec-4d0d-a3ab-a8edd1721f68 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.349014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.349691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.350018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.350287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.360956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.362393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:52.380062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:52.381037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:52.381906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:52.381906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:52.381906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:52.384107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 293/579] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:52.384446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.397212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae84b3c-444e-4253-b2fa-6b93e6828973 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.432797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.433457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.433593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.433859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.434497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.434828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.436204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.445026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.453249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 287/580] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.453249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.505579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6d9c9-a36b-4581-83cd-972206b0548d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.525666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.543976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.545048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.545336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.545607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.554458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.556434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.566195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac8a9d-5028-4722-aaf7-10843c7fd38d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:52.569111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:52.570039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 294/581] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:52.571069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.607617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.608246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.608523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.608793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.619621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.621068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:52.636643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:52.637586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:52.638745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:52.638745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:52.638745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:52.638745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 288/582] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:52.638987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.679399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166b5a6-d7bf-4308-8f0b-de412b2f127a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.695570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.720830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.721575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.721949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.722333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.723068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.723544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.731692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.745649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 295/583] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.746306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.752294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e245-376b-4dc3-8616-390fa1349d84 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.773268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.790022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.790651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.790929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.791258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.800430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.801948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:52.814169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:52.815117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 19:31:52.816100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:52.819960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 289/584] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:52.820703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.857739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d70bd-5bc2-4c61-882e-a0f79db73194 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.896734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.897384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.897662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.897930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.898680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.898986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.900507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.922937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 296/585] 10.222.0.55 () {56 vars in 1137 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.928176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:52.929317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce62be-89ad-4e58-b12f-29e2e19b2a6b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:52.962032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:52.966134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:52.967147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:52.967436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:52.967742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:52.968492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:52.968794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:52.970154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:52.988061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 290/586] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:52.988061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.031193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.036410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181f622-d925-4b77-9698-a3a1a5b58602 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.083757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.084381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.084665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.084937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.092165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.093603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.094632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b1f3-2bb4-4900-9b88-4a3fe4fbf05d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:53.104998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:53.106082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 297/587] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:53.107992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.136478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.137093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.137371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.137641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.138437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:53.138738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.140193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.157764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 291/588] 10.222.0.55 () {56 vars in 1137 bytes} [Sat Sep 24 19:31:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:53.163016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.207380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.217357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4197852-fb81-4cd2-90ed-8aeb17f0dc5d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.257628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.258266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.258540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.258809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.262516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.266734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcd196-e10f-4087-b83f-da7206421bf2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.271345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.272830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:53.288461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:53.289554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:53.290429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:53.290429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:53.290429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:53.290429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 298/589] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:53.290429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.307524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.308161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.308436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.308705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.316446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.317884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:53.329377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:53.330321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 292/590] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:53.331283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.406381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758d518-19da-404b-b540-b5d3b3a6edb7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.437164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4298c-7683-4225-83f4-bc887a79e33e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.441612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.442212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.442485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.442751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.443445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:53.443769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.445171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.458134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 299/591] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:53.458943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.461563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.478357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.478931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.479216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.479522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.492107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.493562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:53.508623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:53.509559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:53.510440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:53.510440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:53.510440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:53.510440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 293/592] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:53.510676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.526619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.576531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acef8a-0dee-4055-a3d6-f16f6a0b3879 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.615549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.616154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.616433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.616701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.617335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:53.617635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.617731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efcff1-f152-4fb5-9fb4-dba246c2d13b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.619123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.636231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 300/593] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:53.636231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.656599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.657199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.657473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.657737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.658459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:53.658761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.660216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.677930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 294/594] 10.222.0.55 () {56 vars in 1139 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:53.678739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.711790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.754101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dd631-1048-4af6-8da2-b32af9414247 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.757280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:53.789278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.790211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.790493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.790789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/57f68f991883493dbf78c0bcfd827c1d/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.795283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a18d27-aab0-48c8-b0f1-19de0ca8c8cf None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.809134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=57f68f991883493dbf78c0bcfd827c1d, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.810673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.824757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 301/595] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:53 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/57f68f991883493dbf78c0bcfd827c1d/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:53.825577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.852846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:53.853874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:53.854391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:53.854921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:53.855836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:53.856887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:53.859302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:53.873409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 295/596] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:53.874737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:53.978055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466604e-98df-460a-8f8d-4363c7a9b58a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:53.996009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.000374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7467-f5cc-45ff-910a-afd8e35d3b0f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.030180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.031346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.031346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.031582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.038448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.040544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.042158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.055983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.056645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.056925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:54.057123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:54.057950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:54.058936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:54.059474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.064454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 302/597] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:54.068927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.079018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.081478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:54.101080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:54.102143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:54.103083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:54.103083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:54.103083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:54.109840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 296/598] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:54.109840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.194120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac358d-2d36-4858-8332-d6f304846cd5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.228906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412349c-107a-4669-8cda-38c2d1d251b1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.230411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.231035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.231312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.232267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.239264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.240739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:54.251880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:54.252929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 303/599] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:54.253818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.272878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.273493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.273768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.274041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.274724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:54.275028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.276758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df18b2c-2c57-4a96-835f-644327f3064f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.289572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.291350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 297/600] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:54.292540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb335-9da6-465d-98a1-520955e3a498 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.292824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb335-9da6-465d-98a1-520955e3a498 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.293091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb335-9da6-465d-98a1-520955e3a498 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.298943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 298/601] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:54.300294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb335-9da6-465d-98a1-520955e3a498 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.346113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.367655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ad026-dcf1-41de-bf7d-632712b78c36 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.401961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.402562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.402834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.403095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.404444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:54.404740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.406101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.412746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb335-9da6-465d-98a1-520955e3a498 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.423943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 304/602] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:54.423943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.454830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.455444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.455734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.456023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.456680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:54.456975 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.458664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004c34-651d-4b01-a131-e333398cb859 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.476322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 299/603] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:54.476322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.476322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.476322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.492847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:54.494108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:54.524332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.535080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1475e-6ed2-44bf-8183-36f825f6aabb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.569579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.570167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.570500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.570716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/b93ea821819d462097b1e76ce1ffae3f/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.571860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 300/604] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:54.579721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.598884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=b93ea821819d462097b1e76ce1ffae3f, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.601131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.604779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.624863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 305/605] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:54 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/b93ea821819d462097b1e76ce1ffae3f/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:54.625802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.779179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0840c-e21c-4b39-abf5-486e7f97c9c3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.839542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9deffd-d51e-4424-b37f-10ad90046f1d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:54.853297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.853946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.854278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.854553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.864861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.866328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:54.879846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:54.880746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:54.881611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 301/606] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:54.887086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:54.907834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:54.908851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:54.909322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:54.909781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/6e9ce9098e884f4e9119aeccdbafd4ae/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:54.923002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:54.928366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=6e9ce9098e884f4e9119aeccdbafd4ae, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:54.930664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:54.949366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 306/607] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:54 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/6e9ce9098e884f4e9119aeccdbafd4ae/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:54.951159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:55.026090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:55.040059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c35d9-eb05-4d9e-9d36-7ac4be3090d8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:55.108700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:55.109697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:55.110170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:55.110631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:55.111584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:55.112103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:55.114500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:55.130272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 302/608] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:55.135943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:55.198283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bddf8a-0cc4-4461-893e-f70c89a0ee6c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:55.252325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:55.252960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:55.253233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:55.253500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:55.265571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:55.270584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:55.271620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:55.272469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 307/609] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:55.283311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641ca90-c491-424a-ac8e-ee5f23ef486b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:55.286966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:55.350789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:55.357553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:55.358519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:55.358847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:55.359139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/8702de7dc5fd4fdaa05449938d461de3/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:55.389748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=8702de7dc5fd4fdaa05449938d461de3, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:55.391736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:55.413157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:55.423137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 303/610] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:55 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/8702de7dc5fd4fdaa05449938d461de3/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:55.456174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e119-b94a-4e67-8f2a-4191b8cc702e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:55.505902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:55.506542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:55.506907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:55.507185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:55.507965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:55.508276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:55.509708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:55.523525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 308/611] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:55.530445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:55.649830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15544-d91d-4130-b8b5-7f0d2872385e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:55.690271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:55.690961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:55.691264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:55.691612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:55.702774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:55.704273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:55.723744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:55.724723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:55.725626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:55.725626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:55.725626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:55.725626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 304/612] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:55.730644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:55.849501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519482eb-42d9-47e9-97d3-e21b7c2b7dcc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:55.864516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:55.889392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:55.890061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:55.890339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:55.890606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:55.891253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:55.891679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:55.893058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:55.911867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 309/613] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:55.917493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:55.975878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:56.041742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57f3b-ae46-4840-a955-36a730d7a669 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:56.083785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:56.084468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.084756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.085029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:56.093967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:56.095473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:56.107321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:56.108271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:56.109219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 305/614] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:31:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:56.118872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:56.253266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1997c82-50bd-49e7-97ad-699fb3124e1a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:56.306229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:56.306900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.307208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.307461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:56.308244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:56.308552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:56.310992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211cfa9-59d8-4195-89f5-6a681e4b566b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:56.329235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 310/615] 10.222.0.55 () {56 vars in 1135 bytes} [Sat Sep 24 19:31:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:56.335771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:56.379587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86424ce-5b16-4f1c-ba4e-29c324f3dddb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.379936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86424ce-5b16-4f1c-ba4e-29c324f3dddb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.380141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86424ce-5b16-4f1c-ba4e-29c324f3dddb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:56.381782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 311/616] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:56.386896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.387178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.387466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:56.434113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:56.437523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:56.493481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:56.500420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0890-2448-428e-88bd-263a683cf926 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:56.524844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:56.564869 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:56.565554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.565773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.566051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:56.578468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 312/617] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:56.585146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:56.586665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:56.602116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:56.603115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:56.604047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:56.604047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:56.604047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:56.604047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 306/618] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:56.612627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:56.744310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec63d0-b60a-4baf-9fb2-7c970d0c1f4d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:56.792405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:56.792940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.793219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.793490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:56.794829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:56.795245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:56.796645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:56.814898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 313/619] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:56.820670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:56.948378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd6907-5316-47c3-b465-38d11fb9f274 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:56.955764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:56.990260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:56.990895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:56.991236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:56.991541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.007174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:57.008688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:57.010547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:57.028771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:57.029754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:57.030644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 307/620] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:57.036014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:57.095022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:57.148109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b59bc-5327-496c-ae65-fe860ad8eff2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:57.219312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:57.219978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.220249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.220508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.239988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:57.246054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:57.247708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:57.248718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 314/621] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:31:57.253614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da5857-4563-459a-b7d1-c25d46cb747d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:57.256332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:57.292401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:57.293396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.293954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.294427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.295503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:57.296047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:57.298127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d50e-2c1d-4be3-b39d-f7b4ee526fbe None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:57.319067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 308/622] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:57.323294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45119868-fdfe-414b-9889-03b8ec5a01c0 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.323294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45119868-fdfe-414b-9889-03b8ec5a01c0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.323294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45119868-fdfe-414b-9889-03b8ec5a01c0 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.325814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 309/623] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:57.326613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45119868-fdfe-414b-9889-03b8ec5a01c0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:57.383061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:57.406415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e91e6e-c46b-4ca5-8c99-5f411a33c7fb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:57.428816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:57.448646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45119868-fdfe-414b-9889-03b8ec5a01c0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:57.454642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:57.455657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.456137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.456606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.458074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:57.458579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:57.461310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237e6e1-a586-460b-885f-9aab95c02140 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:57.472488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 315/624] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:57.473972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0628c7-7e71-403f-a9b0-d43655961dff None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.474456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0628c7-7e71-403f-a9b0-d43655961dff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.474963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0628c7-7e71-403f-a9b0-d43655961dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.491787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:57.492518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.492792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.493060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/c72b7d859e954054848d405afa0b466f/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.498032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cb0628c7-7e71-403f-a9b0-d43655961dff None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:57.499489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cb0628c7-7e71-403f-a9b0-d43655961dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:57.509908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=c72b7d859e954054848d405afa0b466f, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:57.511561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:57.539458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 310/625] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:31:57 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/c72b7d859e954054848d405afa0b466f/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:31:57.539458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:57.641773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 316/626] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:57.644149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0b0bb-6985-47e3-a2ec-087cabc91ffe None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.644657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0b0bb-6985-47e3-a2ec-087cabc91ffe None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.645153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0b0bb-6985-47e3-a2ec-087cabc91ffe None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.647038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 317/627] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:31:57.652345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.652990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.653527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.674957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:57.677585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:57.696178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafb99f-1c53-4164-a0e8-a62530304a0c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:57.769223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:57.770177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.770177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.770373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.773007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:57.785084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:57.786960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:57.807959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:57.808967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:57.810500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:57.810500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:57.810500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:57.813188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 311/628] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:57.820185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:57.821233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 318/629] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:57.928973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3ae17-ada4-49ed-bdee-5a1bb2f1c4b2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:57.964609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:57.965541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:57.966482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:57.966899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:57.967743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:57.968223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:57.969773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:57.983182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 312/630] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:57.992514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.000553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.014358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.077582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.119312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0723-39f3-4948-83df-8de789e91023 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.134188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495182f-74d5-4ab9-93e8-d65a4f92037f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.158751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.159722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.160000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.160267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.168329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.170489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.170807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.171468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.171747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.172015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.182988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:58.183293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:58.184259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:31:58.185839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:58.190340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:58.191948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:58.192787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:58.193440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 319/631] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:31:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:58.193512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 313/632] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:58.193581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.212003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.265943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.315096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad9564-2706-4957-a174-5832fef3427e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.350761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b663a-64e5-4bce-b61f-3f68f8285205 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.358306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.359085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.359395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.359699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.360545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.361970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:58.362287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.364989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.393185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 320/633] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:58.393518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.393728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.393884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.394069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.394635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.395302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:58.395627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.398368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.408966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 314/634] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:31:58.409800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.508030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b0eb6-0d35-48d9-aa3b-f9227ecd44c3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.519147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee9b0-5ec1-44f9-aa1e-85767683b276 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.535728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.549639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.550265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.550537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.550804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.555840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ccb-c9f6-4199-9dd4-843c5415cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.556664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ccb-c9f6-4199-9dd4-843c5415cb47 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.557146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ccb-c9f6-4199-9dd4-843c5415cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.557612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ccb-c9f6-4199-9dd4-843c5415cb47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.562161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:58.566962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:58.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:58.568926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:58.573012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 321/635] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:58.573867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.598334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.639039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125ccb-c9f6-4199-9dd4-843c5415cb47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.640421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125ccb-c9f6-4199-9dd4-843c5415cb47 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.688979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb952c30-fefd-408c-a96c-953aceaceae2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.726612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.727371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.727742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.727932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.739863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.741417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.751478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 315/636] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:31:58.751478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113e0da-4906-4a09-9250-7beaccaa19e5 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.751478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113e0da-4906-4a09-9250-7beaccaa19e5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.751478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113e0da-4906-4a09-9250-7beaccaa19e5 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.751478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 316/637] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:31:58.751478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113e0da-4906-4a09-9250-7beaccaa19e5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:58.763740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:58.764777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:58.765676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:58.765676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:58.765676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:58.765676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 322/638] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:58.765676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:58.802309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.827846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:58.861783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113e0da-4906-4a09-9250-7beaccaa19e5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.880504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5be7e-0954-4550-8f6a-ecc9f4ccece3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:58.907152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.907830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.908106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.908370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.909012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:58.909409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.910965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6580a-73a2-498b-83cb-263823ce6f31 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.924915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 317/639] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:58.924915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.924915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.924915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.932854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:58.933483 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:58.933761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:58.934026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:58.942057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:31:58.943236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:31:58.944786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:58.946215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:58.961120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:58.962082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:58.962989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:58.962989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:58.962989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:58.962989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 323/640] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:58.962989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.023532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 318/641] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:31:59.023532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.077806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.096872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864605-429a-4169-bd23-8d30efb8d77b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.100986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.137268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.139006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.139452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.139982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.140876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:59.141354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.143133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.146939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a185-03d2-4ea7-a7e8-db45da673356 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.158294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 324/642] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:59.159476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.184809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.185489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.185770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.186042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.202107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.203772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:31:59.225058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:59.226339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:31:59.227320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:31:59.227320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:31:59.227320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:59.227870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 319/643] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:59.229108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.280121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015aa9-fc3d-43ff-8be6-e6db58b8e748 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.321910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.322567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.322842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.323129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.323826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:59.324131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.326782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.340221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 325/644] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:59.341392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.350978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.356121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb553-54d8-47fc-9d47-4e1f1ac3b4b4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.369291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.393172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.393806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.394085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.394356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.402017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.403522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:59.414888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:59.416031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 320/645] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:59.417140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.459099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552aeaa-8ec9-4257-b785-88298ca09819 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.494631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.495236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.495549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.495826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.496465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:59.497275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.498979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.516420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 326/646] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:59.516420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.534659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84ed9-5b64-4b35-ad66-7b5375a1e16c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.570758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.571383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.571696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.572044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.581713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.583734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:31:59.589059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:59.590053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 321/647] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:59.596309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.623620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab0750-e8d7-4b22-bd17-6a5e45b24f08 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.627373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.661165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.661798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.662118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.662342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.670008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.671576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:31:59.684158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:59.685868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 327/648] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:59.686788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.711353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d261b5e-9854-4e59-bf74-aed37057c66e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.745654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.746414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.746548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.746911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.748238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:59.748465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.749896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.772154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 322/649] 10.222.0.55 () {56 vars in 1137 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:59.772154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.803768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7f31d-e095-4fc4-ad84-c6fa3dfd44db None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.830997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.841362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.842171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.842454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.842849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.850948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.852436 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.866855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:31:59.867694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:31:59.869189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:31:59.870173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 328/650] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:31:59.870802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.883055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0139-2697-45e0-8772-1d1b86572658 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:31:59.921392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:31:59.922192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:31:59.922474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:31:59.922742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:31:59.923419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:31:59.923781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:31:59.925193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:31:59.939800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 323/651] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:31:59.939800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:31:59.986860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e01782-8f56-4d11-aabe-3a997f6bf517 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.023096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.023734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.024011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.024280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.024969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:00.025271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.026694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.039608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 329/652] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:00.040393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.050478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3cb03-a9c0-43ae-9dbd-67c0a2b2803d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.086844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.087085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35284c-b408-4b7c-90a2-c6f953613efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.087821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35284c-b408-4b7c-90a2-c6f953613efa None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.088078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35284c-b408-4b7c-90a2-c6f953613efa None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.088314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35284c-b408-4b7c-90a2-c6f953613efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.110623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.142339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc0845-c6e4-480d-a082-75d768c8504d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.155744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35284c-b408-4b7c-90a2-c6f953613efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.157199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35284c-b408-4b7c-90a2-c6f953613efa None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.177598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.178206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.178481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.178746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.179738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:00.180148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.181452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.198379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 330/653] 10.222.0.55 () {56 vars in 1137 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:00.199107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.261356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 324/654] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:32:00.263123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb109da-759a-4558-8a8f-89a9ad6addb7 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.263395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb109da-759a-4558-8a8f-89a9ad6addb7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.263697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb109da-759a-4558-8a8f-89a9ad6addb7 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.264985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 325/655] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:00.265679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb109da-759a-4558-8a8f-89a9ad6addb7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.308387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7578b-ce6d-4ca3-afeb-a2297c8f6463 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.310724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.320968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.343028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.343683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.343962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.344234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.354115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.355585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.369160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb109da-759a-4558-8a8f-89a9ad6addb7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:00.371119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:00.372729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:00.373630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:00.373630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:00.373630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:00.373630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 331/656] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:00.373630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.407863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.408459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.408732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.409000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.409709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=57f68f991883493dbf78c0bcfd827c1d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:00.410011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.411483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.417021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 326/657] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/role_assignments?user.id=57f68f991883493dbf78c0bcfd827c1d&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:00.417832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.486119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1db5c-fb17-41f4-9114-ee18039cb6b7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.527128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.527947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.529399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.529680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.540103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857410a3-6577-4e3c-8ee6-9b9a3eb0cc5c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.561558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.572467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.587800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.588441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.588724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.588998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.602288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.603478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-9d9b0a86-48c4-4449-b92d-2a42a84e63fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:00.623588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:00.624578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:00.625483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:00.625483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:00.625483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:00.625483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 327/658] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:00.625921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023eb21-e58f-448a-acd0-476108711537 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.627457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023eb21-e58f-448a-acd0-476108711537 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.627457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023eb21-e58f-448a-acd0-476108711537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.633217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.634933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.644531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5023eb21-e58f-448a-acd0-476108711537 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:00.645716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5023eb21-e58f-448a-acd0-476108711537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:00.719963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 328/659] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:00.720848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023eb21-e58f-448a-acd0-476108711537 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.750822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 332/660] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:32:00.752204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.818951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.860114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:00.867288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023eb21-e58f-448a-acd0-476108711537 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.876039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636e75b-9fe5-4fd7-9407-29ccb28676f1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:00.916200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.916965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.917288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.917631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.918362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:00.919372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.920823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:00.921118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.921907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:00.922368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:00.923084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:00.924656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:00.925168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:00.926729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:00.939377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 333/661] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:00.940481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:00.947340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 329/662] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:00.947340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.067046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94db73-dd52-41f5-89a6-9c9a516b7160 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.077041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0a335-7896-437b-9402-ff9f904a5424 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.087823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.109426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.110446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.110979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.111256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.115296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.115972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.116252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.116522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.122002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.123457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.123725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.125170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:01.136512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:01.137430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 330/663] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:01.138992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:01.143245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:01.144340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:01.146350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:01.146350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:01.146350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:01.146350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 334/664] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:01.146350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.164189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.255900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39517b83-30c8-4891-bf1e-37c8d69728ca None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.259808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb2b9-1dfd-4e03-8da6-df61d72f7693 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.292847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.293476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.293778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.294046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.295858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.296488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.296891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.297059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.304917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:01.305073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.306388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:01.310262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:01.311779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:01.312857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:01.315769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 335/665] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:01.316878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:01.319445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:01.320362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 331/666] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:01.321297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.350783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.412067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.431559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b74d-6a09-4316-b496-efc0e5f1dff7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.435594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674923-db23-4e50-ae20-576fc6401fa4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.472533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.473360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.473792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.473899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.474522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:01.474820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.476224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.478981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.479617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.479895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.480159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.480841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:01.481690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.483196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.493135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 332/667] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:01.493135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.497799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 336/668] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:01.498834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.594945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.595296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195ea3-1f85-4c2f-9e73-ee6194925360 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.609021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369a784-bd35-4ca7-ad38-e9b3d4e58d72 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.630698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.631305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.631621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.631888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.632573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:01.632873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.634233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.639892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.643682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 333/669] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:01.644568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.654365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.655164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.655981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.656412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.657584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:01.658288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.659884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.683084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 337/670] 10.222.0.55 () {56 vars in 1139 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:01.683084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.751138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7d15d-8f79-4cea-8597-000e0c1d5807 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.785254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.785884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.786171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.786458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.786558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0cfa-2d4d-43be-b076-b7d390ae2292 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.793218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.794937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:01.806445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:01.807360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 334/671] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:01.808301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.828317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:01.828923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:01.829227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:01.829468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:01.830182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=b93ea821819d462097b1e76ce1ffae3f) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:01.830546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:01.831993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:01.847452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 338/672] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/role_assignments?user.id=b93ea821819d462097b1e76ce1ffae3f&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:01.847452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:01.861378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.893272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:01.953919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e35ba-fb19-48c3-bf8c-a77e246fa299 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.973591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2685980-28e7-4284-99a5-d447c44d1793 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:01.999520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.000494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.000777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.001048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.014207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.014974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.015353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.015750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.016839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.018323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:02.039346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:02.040555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:02.041722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:02.041722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:02.041722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:02.044116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 335/673] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:02.046123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.112562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.113943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.129551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.180824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94453c1d-20fd-4157-b8be-6f14b4f346b1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.195974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.223986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.225204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.225956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.226816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.228090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:02.228595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.230258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.230386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 339/674] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:32:02.231858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.248611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 336/675] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:02.249971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.359840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6608f93-f9e5-41d5-b618-9348a322918d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.364571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268801e6-5d1e-4112-99a3-b94cf2c3b8ed None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.373527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.401846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.402448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.402722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.403576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.404028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.404213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:02.404510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.405138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.405568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.405870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.406126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.417152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.419558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.420319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 340/676] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:02.421398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:02.436909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:02.437912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:02.438820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:02.438820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:02.438820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:02.445306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 337/677] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:02.446372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.474872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.538446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a53963-6c13-4086-8362-bd0aa367b371 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.564526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa4ae-0553-4007-a091-cc702e78c4cf None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.578391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.579406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.579846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.580271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.581155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=6e9ce9098e884f4e9119aeccdbafd4ae) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:02.581609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.583369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.599864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 341/678] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/role_assignments?user.id=6e9ce9098e884f4e9119aeccdbafd4ae&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:02.601145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.616263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.616994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.617396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.617766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.629180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.636913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.640662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:02.660503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:02.661710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:02.662904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:02.664728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 338/679] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:02.671187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.736361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc37d-94a6-4be1-af69-d660e2281567 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.780124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.781052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.781479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.781870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.782526 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:02.782823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.784241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29895c66-4bdf-47c1-99f0-232b6f156778 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.798802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 342/680] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:02.800237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e98c-de92-4943-9f05-df58fa339822 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.800522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e98c-de92-4943-9f05-df58fa339822 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.800789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e98c-de92-4943-9f05-df58fa339822 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.807203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 343/681] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:02.808676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e98c-de92-4943-9f05-df58fa339822 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:02.816240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc62fc-69b7-4adc-91fb-2642c6dbbe1a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.819769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.866808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.867663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.868037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.868537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.869741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:02.870269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.871881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704e2a8-e230-4289-a902-4fe6b4bb7ba5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:02.892370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 339/682] 10.222.0.55 () {56 vars in 1135 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:02.893664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.894219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.894355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.922508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:02.923503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:02.931454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:02.941814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e98c-de92-4943-9f05-df58fa339822 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:02.980428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:02.981088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:02.981406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:02.981709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:02.989778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:02.991210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:03.003561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:03.004472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:03.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 344/683] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:03.005862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.020168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 340/684] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:03.021235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.164802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b63d-4e7e-48de-8519-f7093eeb7ddf None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.170276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:03.203900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fa98f-6ec2-4bd0-8c92-a16eeed4afc8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.240204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.240811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.241083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.241351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.242977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.243602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.243943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.244211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.244888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:03.245183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:03.246622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.271692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 341/685] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:03.277253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.287278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:03.363261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:03.364754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.422741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3885f-003a-4a37-a58e-c1ba9af4ae9c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.455396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:03.461603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.462478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.462976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.463788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.464809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=8702de7dc5fd4fdaa05449938d461de3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:03.465590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:03.467241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.474106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 342/686] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/role_assignments?user.id=8702de7dc5fd4fdaa05449938d461de3&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:03.475330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.483359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 345/687] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:32:03.488825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.553656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:03.609235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7e165-ac27-4f30-8016-788837742574 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.610977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c126-80d6-4abf-a360-18e932df215b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.660752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.661800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.662280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.662770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.696670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:03.708259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:03.710003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:03.710958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:03.714898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 343/688] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:03.716124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.717132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.717681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.718165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.725148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.751712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:03.753781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:03.781276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:03.782266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:03.783176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:03.783176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:03.783176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:03.785570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 346/689] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:03.791172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.810802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:03.870194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933606af-c616-4ac8-8af9-b7e10e2f0d3b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.912464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd757a8-95c6-4f4d-857c-4877c2b81ae9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:03.913208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.914766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.915266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.915829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.916786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:03.917593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:03.920844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.922162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:03.935302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 344/690] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:03.942598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:03.960906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:03.961613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:03.961886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:03.962649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:03.963306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:03.963788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:03.965495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:03.989549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 347/691] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:03.995720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.065364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d686-9ac1-46c7-b7a6-3918d383ce33 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:04.088764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:04.130513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cdd0-09de-4261-85a2-c4dce6a67df5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:04.131523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:04.132280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.132678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.133020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.154857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:04.156629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:04.174081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:04.175140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.175657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.176229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:04.182778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:04.183803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:04.185702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:04.185702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:04.185702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:04.188436 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:04.190461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:04.192666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 345/692] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:04.193793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:04.216148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:04.217144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:04.218048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:04.222868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 348/693] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:04.230424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.297461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:04.394844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adcf99-84e1-4e48-ade3-057b5576584f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:04.436594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64642-2cd7-4d4a-ba37-ab8a7f41a055 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:04.438494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:04.440081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.440396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.440665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.441256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:04.441548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:04.442928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20edcd2-4221-4233-85da-c9a285c58df5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:04.460098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 346/694] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:04.461910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020dcb1-2605-4b04-8cb8-434fab203b2a None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.462188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020dcb1-2605-4b04-8cb8-434fab203b2a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.463748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020dcb1-2605-4b04-8cb8-434fab203b2a None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.470026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 347/695] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:04.470740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020dcb1-2605-4b04-8cb8-434fab203b2a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.484765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:04.485372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.485643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.485906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.486619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:04.486913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:04.488358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cceac-4ee4-42a1-a411-f50d465dd164 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:04.508561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 349/696] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:04.511133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.511414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.511700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.521048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:04.538615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:04.540229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:04.615849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020dcb1-2605-4b04-8cb8-434fab203b2a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:04.628798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 350/697] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:04.630364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.659633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:04.684723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:04.686542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.689544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.690460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.718658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:04.727018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:04.744495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:04.745499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:04.746446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:04.755065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 348/698] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:32:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:04.761707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.831104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafabcb-4689-4d80-8555-af4cb836ae66 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:04.905475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:04.906088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:04.906364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:04.906628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:04.908177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=c72b7d859e954054848d405afa0b466f) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:04.908489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:04.909938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:04.923738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 351/699] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:04 2022] GET /identity/v3/role_assignments?user.id=c72b7d859e954054848d405afa0b466f&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:04.959017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:04.969482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:04.989439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcb02a-ff7e-48e3-a7dd-82b61d83beec None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.069977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.070655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.070927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.071245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.072010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:05.072310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.073755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.120319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 349/700] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:32:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:05.126171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:05.136899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:05.166783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb1ce6-c3f5-4482-ba6e-a77399cca8aa None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.207281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.207967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.208246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.209068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.209666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.211532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:05.226832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:05.227984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:05.228835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 352/701] 10.222.0.55 () {56 vars in 1129 bytes} [Sat Sep 24 19:32:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:05.234092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:05.264658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1d6d5-8f05-4ccf-a98c-c7dfe9132735 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.304692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81fe94-e194-4b2d-8e40-641e557fda88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.305290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81fe94-e194-4b2d-8e40-641e557fda88 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.305705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81fe94-e194-4b2d-8e40-641e557fda88 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.306136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81fe94-e194-4b2d-8e40-641e557fda88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.328157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:05.351136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d6be-dfec-46eb-b436-30d672cd0ef8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.390455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81fe94-e194-4b2d-8e40-641e557fda88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.391318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.392111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.392383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81fe94-e194-4b2d-8e40-641e557fda88 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.392708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.393129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.393899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:05.394351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.395900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.405998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 353/702] 10.222.0.55 () {56 vars in 1124 bytes} [Sat Sep 24 19:32:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:05.407453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:05.418496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:05.503642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 350/703] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 19:32:05.505290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cdfb-2928-4b43-8f7d-37301068446f None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.505675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cdfb-2928-4b43-8f7d-37301068446f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.506062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cdfb-2928-4b43-8f7d-37301068446f None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.508279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 351/704] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:05.509757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cdfb-2928-4b43-8f7d-37301068446f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:05.519010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99eb4-e8c4-4dae-a048-841611ebf7ef None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.560319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.561050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.561330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.561598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.562219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:05.562513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.563968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560edb-a67c-4f25-8137-167c5f6afb17 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.565862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:05.571279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 354/705] 10.222.0.55 () {56 vars in 1124 bytes} [Sat Sep 24 19:32:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:05.574803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.574803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.574803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.614625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:05.614625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:05.670009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cdfb-2928-4b43-8f7d-37301068446f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.723849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.724498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.724775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.725041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.738013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.739522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.750512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 355/706] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:05.756482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:05.757500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:05.758419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:05.758419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:05.758419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:05.758419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 352/707] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:05.763302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:05.766249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:05.889052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b0ca-2a32-438c-9d37-2d4a0a6e421c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:05.925280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:05.925882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:05.926153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:05.926420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:05.927056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:05.927350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:05.928793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:05.941960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 356/708] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:05.947033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.109413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:06.112414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402b54-a5b0-42b8-a079-95f069ed1ad0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:06.170949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.173249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:06.173902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.174182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.174459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.189912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:06.191377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:06.214196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:06.215151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:06.216721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 353/709] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:06.224076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.299371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:06.342337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea20d4d-cd0a-48d0-a76a-12b8edbbc4c5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:06.380785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:06.381802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.382221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.382629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.395902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:06.397060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8f654-6c80-4a51-bed5-3c4285940d6e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:06.397740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d9b4c31d-9c44-4355-9f40-8baa38391f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:06.416055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:06.417231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:06.418321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:06.418321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:06.418321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:06.421195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 357/710] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:06.423281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af43b4-5f53-49bd-8ded-94e04b72eb6b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.423971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af43b4-5f53-49bd-8ded-94e04b72eb6b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.424390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af43b4-5f53-49bd-8ded-94e04b72eb6b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.426298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 358/711] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:06.427127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af43b4-5f53-49bd-8ded-94e04b72eb6b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.434517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:06.435244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.435653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.436016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.436795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:06.437194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:06.438718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755787-0262-4b65-8c66-0c9af8c8e348 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:06.452621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 354/712] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:06.457668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.457668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.457668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.474786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:06.476191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:06.491880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:06.547634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 355/713] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:06.556198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.556946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af43b4-5f53-49bd-8ded-94e04b72eb6b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:06.583789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:06.624825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:06.625712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.630631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.631057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.631887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:06.632330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:06.633849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:06.653642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 359/714] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:06.658440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.741184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fc24e-6803-49ee-9bce-eb382c6f4fbb None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:06.811193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:06.812110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.812537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.812955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.813825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=57f68f991883493dbf78c0bcfd827c1d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:06.814281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:06.815921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:06.829762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 356/715] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:06 2022] GET /identity/v3/role_assignments?user.id=57f68f991883493dbf78c0bcfd827c1d&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:06.841560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:06.862058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:06.863622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceaa99-4a0a-44a7-81f3-8bcfccb4a13c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:06.963954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:06.965136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:06.967046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:06.967779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:06.969666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:06.973102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:06.990649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aec3-6fc0-4fe6-b008-dd990b5eca1a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:07.018101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 360/716] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:32:06 2022] POST /identity/v3/users => generated 254 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:07.054386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:07.055751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:07.056264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:07.056741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:07.057642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:07.063453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:07.092827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:07.093915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:07.094787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 357/717] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:06 2022] GET /identity/v3/services/network => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:07.100323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:07.151876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:07.232630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231cf86-c964-4f93-808c-9691cefa0cbb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:07.270952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:07.271619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:07.272006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:07.272347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:07.273058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] RBAC: Adding query filter params (name=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:07.273428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:07.274873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064f818-ef9f-41ad-b57d-9a304fcd93ce None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:07.282151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 361/718] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:07.287967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:07.381803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:07.409588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e7dd5-9f92-4e48-b0fc-a6f310fa79f1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:07.452443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:07.453428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:07.453928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:07.454393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:07.455215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] RBAC: Adding query filter params (type=network) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:07.455739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:07.457300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c465b-f1ce-4867-b830-2e1e2d90cc16 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:07.464189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 358/719] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:07.885017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7caed7-6307-49bf-a9ab-f06c11baae7c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:07.885555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7caed7-6307-49bf-a9ab-f06c11baae7c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:07.885844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7caed7-6307-49bf-a9ab-f06c11baae7c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:07.887553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 362/720] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:07.893331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc1ed-2df9-4641-b6cf-64711ac5c236 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:07.893620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc1ed-2df9-4641-b6cf-64711ac5c236 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:07.893893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc1ed-2df9-4641-b6cf-64711ac5c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:07.919338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ff3fc1ed-2df9-4641-b6cf-64711ac5c236 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:07.920632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ff3fc1ed-2df9-4641-b6cf-64711ac5c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:08.003732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 359/721] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:08.140475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62a391-6ae6-4f8c-a097-ce49adb671fb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:08.140701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62a391-6ae6-4f8c-a097-ce49adb671fb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:08.140936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62a391-6ae6-4f8c-a097-ce49adb671fb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:08.142530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 363/722] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:08.147817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:08.148108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:08.148384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:08.167309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:08.169183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:08.278003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 360/723] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:08.384932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62a391-6ae6-4f8c-a097-ce49adb671fb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:08.590658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:08.605033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62a391-6ae6-4f8c-a097-ce49adb671fb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:08.664129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:08.664775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:08.665052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:08.665322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:08.673434 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:08.675799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:08.690800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:08.695591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:08.696523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:08.697381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:08.700511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 364/724] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:08.702988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:08.840874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beb242-33d2-4c96-99cf-d2186bbd9d2e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:08.896291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:08.896932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:08.897180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:08.897447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:08.898122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:08.898423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:08.899857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:08.927453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 365/725] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:08.927453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:08.962706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f22411-0b4c-4b3a-a38c-3c7d61deb8d4 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:08.986118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:09.046067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:09.046687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.046961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.047224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.047915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:09.049558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:09.087939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 361/726] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:08 2022] POST /identity/v3/services => generated 251 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:09.142445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae67839-3d3d-475d-be9b-e72e873b7f4f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:09.221084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:09.221740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.222018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.222291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.240059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:09.241577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:09.269154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:09.270187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:09.271085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:09.271085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:09.271085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:09.271257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 366/727] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:09.278553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:09.352322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:09.397307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bfdbc-5351-49e2-b65d-f48c461370e2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:09.435372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:09.436014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.436291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.436557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.437194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:09.437489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:09.438860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f5a2-1d4b-4772-b46f-a585693f1a2d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:09.452653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 362/728] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:09.457889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:09.530225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:09.570135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fa051-bcc9-48cc-9310-0d9504cdfc46 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:09.605655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:09.606310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.606584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.606848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.614917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:09.616376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:09.626671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2d1e-aff5-454e-985f-f660eaf28d9a None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.626970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2d1e-aff5-454e-985f-f660eaf28d9a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.627207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2d1e-aff5-454e-985f-f660eaf28d9a None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.628810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 363/729] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:09.629975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:09.630905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:09.632235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 367/730] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:09.633848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.634132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.634399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.639906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:09.657332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:09.658216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:09.734649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 364/731] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:09.854886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1b25-a653-4646-85da-d2b3ffb7f436 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:09.889104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:09.904710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:09.905748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:09.906243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:09.906607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:09.907578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:09.907736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:09.909191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:09.935994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 368/732] 10.222.0.55 () {56 vars in 1135 bytes} [Sat Sep 24 19:32:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:09.941708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.024046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:10.099693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e74b-f150-44e0-929b-1c6a6762a90f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:10.107328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.160293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b311d5-c15a-47e3-9bce-e107db8ec361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:10.160840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b311d5-c15a-47e3-9bce-e107db8ec361 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.161117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b311d5-c15a-47e3-9bce-e107db8ec361 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.161383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b311d5-c15a-47e3-9bce-e107db8ec361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.281354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b311d5-c15a-47e3-9bce-e107db8ec361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:10.282837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b311d5-c15a-47e3-9bce-e107db8ec361 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:10.346120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:10.361334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a5aa0-15b7-4537-88b5-88bbb443afaf None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:10.400792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 365/733] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:09 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 19:32:10.402417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e82b4-aacb-4d06-8ec3-6d10de06a844 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.403568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e82b4-aacb-4d06-8ec3-6d10de06a844 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.404189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e82b4-aacb-4d06-8ec3-6d10de06a844 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.405666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 366/734] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:10.406831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d82af-0165-4364-b0c0-b9b1b5daaac8 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.407114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d82af-0165-4364-b0c0-b9b1b5daaac8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.407381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d82af-0165-4364-b0c0-b9b1b5daaac8 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.408695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 367/735] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:10.411989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8d1f1-750b-4c8e-9d4e-5f49ec65c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:10.412505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d82af-0165-4364-b0c0-b9b1b5daaac8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.412803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8d1f1-750b-4c8e-9d4e-5f49ec65c873 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.413082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8d1f1-750b-4c8e-9d4e-5f49ec65c873 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.413346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8d1f1-750b-4c8e-9d4e-5f49ec65c873 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.424700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8d1f1-750b-4c8e-9d4e-5f49ec65c873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1ed8d1f1-750b-4c8e-9d4e-5f49ec65c873 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:10.429958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:10.430990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:10.432251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:10.435610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 369/736] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:10.437324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57501f-beaf-4f37-9ef8-b2cb74092ce5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.438310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57501f-beaf-4f37-9ef8-b2cb74092ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.438586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57501f-beaf-4f37-9ef8-b2cb74092ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.457488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dc57501f-beaf-4f37-9ef8-b2cb74092ce5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:10.458688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dc57501f-beaf-4f37-9ef8-b2cb74092ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:10.496411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:10.521035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d82af-0165-4364-b0c0-b9b1b5daaac8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:10.531157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 370/737] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:10.538683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.538683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.538683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.551874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:10.553264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:10.565205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:10.565924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.566265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.566601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.577291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:10.578836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:10.593594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:10.594529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:10.595461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:10.595461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:10.595461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:10.599668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 368/738] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:10.600581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.604108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:10.635879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 371/739] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 19:32:10.637684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.702547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aabaa9-7739-4a22-adf5-f471ec35d18e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:10.738266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:10.739715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.740192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.740670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.741572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:10.742065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:10.743712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:10.752982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 369/740] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:10.764445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.839792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:10.864115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5445be-d75b-4948-8009-5756ae96b1d0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:10.865662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:10.872552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc560b6-cca1-474d-9cdd-225b8dc9279d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:10.901751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:10.902365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.903159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.903709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.904370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:10.904674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:10.906054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:10.917975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 372/741] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:10.919365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:10.920718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:10.921598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:10.922175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:10.923028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:10.935564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:10.937425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:10.953795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:10.954711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:10.955632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:10.955632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:10.955632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:10.957774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 370/742] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:10.963118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.045658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8837d63-370e-4d5a-bc0d-91e4aeec11d4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.088776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.089518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.089806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.090091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.101070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:11.104935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3b38-6af3-4ca8-a676-f858bd618494 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:11.106701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:11.107677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:11.108540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:11.109179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 373/743] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:11.109810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.124195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.136847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.143200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.143851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.144130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.144400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.151199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.154172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:11.166794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:11.172777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 371/744] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:11.173958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.224296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669cac9-99b7-4051-adce-8a69185b9871 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.264883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.265502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.265876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.266165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.266813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:11.267246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.268642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.283357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69b0bd-7da8-457a-9a10-c0beacf0ae03 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.287878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 374/745] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:11.288624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.320884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.321692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.322243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.322531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.323336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.324788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.339488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 372/746] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:11 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:11.340998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.390703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.399129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.422383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb5357-7397-4e05-87f2-e6e7b9ece692 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.469764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4f2a-cb2f-4a73-b3ae-12ff40f75b09 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.470762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.471582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.472095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.472414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.474016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:11.474016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.475204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.490199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 375/747] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:11.491814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.511250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.512768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.513229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.513732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.514636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:11.515087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.516798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.532143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 373/748] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:11.533416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.640393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1a61-d53d-4d4c-840e-4b2f0b3b48d0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.648410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.657022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9b853-82e9-41b3-8397-b4045129de70 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.687240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.688186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.688467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.688737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.698470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.699988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.700312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.700699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.701167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.701791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.705841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.717772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:11.718433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:11.719391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 376/749] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:11.721198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.723121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:11.751637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:11.753358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:11.754325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:11.754325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:11.754325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:11.754325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 374/750] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:11.759006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.838274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a922-0a1f-439b-926f-c29579587707 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.878587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.879486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.879771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.880039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.880764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=c72b7d859e954054848d405afa0b466f) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:11.881070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.881175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c598727-a0e1-41d4-bd9c-acde1039261f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:11.882747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.891598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 377/751] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/role_assignments?user.id=c72b7d859e954054848d405afa0b466f&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:11.892556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.931621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:11.932656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.933192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.933655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.935722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:11.936254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:11.937926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7b0b3-249f-4d0d-a70a-2931a7e180a9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:11.960692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 375/752] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:11.962421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fc7f9-8bcd-464e-bda7-1409a0237cc3 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:11.962899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fc7f9-8bcd-464e-bda7-1409a0237cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:11.963361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fc7f9-8bcd-464e-bda7-1409a0237cc3 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:11.965367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 376/753] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:11.966739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fc7f9-8bcd-464e-bda7-1409a0237cc3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:11.973817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:11.988695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:12.044401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f93b91-5f02-431a-beb0-1f446d3f83d8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.087708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.088508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.088793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.089059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.089709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:12.091007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.099598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089d2d1-4a8b-418c-8edb-fa562a16e241 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:12.105555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 378/754] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:12.107766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.108045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.108315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.121193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fc7f9-8bcd-464e-bda7-1409a0237cc3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.131527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:12.132615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:12.167252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.169121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.170079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.170627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.182913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.185543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:12.207674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 379/755] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:12.208576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:12.272111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:12.327564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd9e8e-a841-461f-8fc0-01f067179d40 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.384169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.384824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.385108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.385381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:12.387111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:12.388381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:12.389340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:12.389340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:12.389340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:12.391481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 377/756] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:12.399455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.400005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:12.406748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:12.419489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:12.420707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:12.422887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 380/757] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:32:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:12.426489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:12.547406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:12.586656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91658c88-e1eb-458a-9587-0949e423be51 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.615337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:12.628267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f32c0e-b482-46a0-960d-c6ca0da8eb65 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.638662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.639274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.639584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.639857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.640722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:12.640923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.642299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:12.661374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 381/758] 10.222.0.55 () {56 vars in 1149 bytes} [Sat Sep 24 19:32:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:12.662227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:12.676837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.678240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.679143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.680050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.681028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:12.681554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.684263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:12.701617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 378/759] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:12.702872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:12.781828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86580ed9-5218-4879-bd1e-c471f5ca314c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.822753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.823412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.823730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.824001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.825130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.826601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:12.836260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94017bc-1fea-4f0a-8277-ab1b1df34fc6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:12.836784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:12.838089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:12.838932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 382/760] 10.222.0.55 () {56 vars in 1129 bytes} [Sat Sep 24 19:32:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:12.839776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:12.873745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:12.893817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:12.904738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:12.905921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:12.906428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:12.907070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:12.984766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:12.985291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4ab9d-6481-4649-bdf5-6436b2ad73ec None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:12.986672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.036707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.038633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.039125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.039862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.049943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.052570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:13.079100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:13.080151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:13.081065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:13.085297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 383/761] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:13.086144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.119363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 379/762] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:32:13.120615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.144325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:13.196588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:13.203759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb5e9d-acc6-474b-9c47-74b9a918f295 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.240751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.241689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.242162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.242624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.243578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:13.244074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.245627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.256411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 384/763] 10.222.0.55 () {56 vars in 1124 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:13.257032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0931-7715-4d78-8b01-0b1b34cc5db6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.257830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.296691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.297461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.297915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.298277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.299713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:13.299713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.301036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.319464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 380/764] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:13.320559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.375961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1340-32d6-4db3-a4c6-4a08efdd09d4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.412703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.413746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.414224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.414684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.427417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.429190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.432694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:13.446433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:13.447467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:13.448374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:13.448374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:13.448374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:13.450540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 385/765] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:13.451969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.470788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9867-a9ca-43c0-84a1-49725ba12bce None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.474705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:13.508179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.509138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.509658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.510127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.511037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:13.511577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.513875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.523574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 381/766] 10.222.0.55 () {56 vars in 1124 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:13.524773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.587314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c1d8-09de-4165-98df-50418c9a1056 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.628780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.630339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.630926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.631413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/57f68f991883493dbf78c0bcfd827c1d/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.633624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=57f68f991883493dbf78c0bcfd827c1d, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.635261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.650674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6427a-0f92-46cc-9a00-1a7dc53279e4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.653684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 386/767] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:32:13 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/57f68f991883493dbf78c0bcfd827c1d/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:32:13.655043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.707597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.708241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.708522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.708803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.709448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:13.709758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.711234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d86a-31fc-4a8a-8649-b5e573ef908e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.723006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:13.726752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 382/768] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:13.732407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c020a-9c47-453b-867b-87d3e61b638f None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.734094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c020a-9c47-453b-867b-87d3e61b638f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.734430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c020a-9c47-453b-867b-87d3e61b638f None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.736178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 383/769] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:13.736973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c020a-9c47-453b-867b-87d3e61b638f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.757092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:13.799627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908da775-95db-46f2-ba04-b0f176f14176 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.853888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.854961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.855250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.855556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.856842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] RBAC: Adding query filter params (interface=public, service_id=869df14f7fea417785d689ada608f906, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:13.857281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.859329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5008b3b-69a3-4416-b34b-be56ba4d966a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.872833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 387/770] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/endpoints?service_id=869df14f7fea417785d689ada608f906&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:13.873050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.873255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.873529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.874952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c020a-9c47-453b-867b-87d3e61b638f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:13.894987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:13.896135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:13.923551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:13.924352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:13.924676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:13.924964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:13.935337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:13.937265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:13.952979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:13.953891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:13.954809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:13.956813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 384/771] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:13.958557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:13.980443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 388/772] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:13.987385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.084016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:14.111088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:14.147271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6908-77d5-4325-8d29-7e718506dc35 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.194238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f2b46-a92b-42f9-8e21-1bc3547c2cbf None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.208659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.209273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.209546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.209811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.210507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:14.210809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:14.212329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367682b9-669b-47fe-a1aa-661a959f98cc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:14.227936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 389/773] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:14.229206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a334b3-105f-4838-a697-c060a7f6f671 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.229486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a334b3-105f-4838-a697-c060a7f6f671 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.229752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a334b3-105f-4838-a697-c060a7f6f671 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.231918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 390/774] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:14.233759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a334b3-105f-4838-a697-c060a7f6f671 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.251795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a84e4c-f672-4d7c-8c2a-5a5d5314daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.252411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84e4c-f672-4d7c-8c2a-5a5d5314daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.252689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84e4c-f672-4d7c-8c2a-5a5d5314daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.253021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84e4c-f672-4d7c-8c2a-5a5d5314daf1 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.253628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a84e4c-f672-4d7c-8c2a-5a5d5314daf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:14.255103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a84e4c-f672-4d7c-8c2a-5a5d5314daf1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:14.265933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 385/775] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:13 2022] GET /identity/v3/services => generated 772 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:14.265933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.265933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.265933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.284600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:14.285947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:14.375454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 386/776] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:14.375454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.431082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a334b3-105f-4838-a697-c060a7f6f671 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.434779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:14.485774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4790f0-c404-4141-a43f-4f5398ebff27 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.501172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:14.512833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.513769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.514351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.514823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.515862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=81fd5bd3dbf743d8b6c6545fa33afdb6) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:14.516389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:14.518197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:14.524614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 391/777] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/role_assignments?user.id=81fd5bd3dbf743d8b6c6545fa33afdb6&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:14.526404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.527269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.528403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.528839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] PATH_INFO: `/v3/services/network` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.529575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:14.531148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:14.545520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:14.546675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:14.547558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:14.550164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 387/778] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:14.556361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.668042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9d12-3f4f-45cd-8892-333da7e14cef None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.708138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.708677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.708953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.709336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.709930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] RBAC: Adding query filter params (name=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:14.710212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:14.711663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:14.718505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 392/779] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:14.722955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.724274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.793673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:14.856567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6bb04-cb79-4f6d-b0a1-4991fc51cb89 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.882527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8085-cdfe-4bff-8db2-7279be9027ef None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:14.904033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.906727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.907002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.907265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.908586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:14.909804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] RBAC: Adding query filter params (type=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:14.913216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:14.914605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:14.930419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 393/780] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:14.936281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:14.943668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:14.944306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:14.944578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:14.944844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:14.967350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:14.974256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:14.975235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:14.976140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 388/781] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:14.982667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:15.125412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29a0f5-486c-4103-87a6-b19ee7649703 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:15.139856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575509c-4c33-47cb-9487-871e6c48b94f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:15.170147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:15.191910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:15.192529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:15.192805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:15.193121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:15.193803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:15.194103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:15.195507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:15.203783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:15.204391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:15.204665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:15.204936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:15.205639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] RBAC: Adding query filter params (interface=public, service_id=2a0f33e75e3c4caebb19ec62cd6891b3, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:15.205945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:15.207373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:15.220956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 389/782] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:15.221812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:15.222421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 394/783] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:14 2022] GET /identity/v3/endpoints?service_id=2a0f33e75e3c4caebb19ec62cd6891b3&interface=public®ion_id=RegionOne => generated 194 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:15.227350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:15.293781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:15.388952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218dbc5-8fe7-4169-ad40-fafdd4a4ea52 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:15.401024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea790b-7908-469f-931b-617fe2e1bac5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:15.437019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:15.437652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:15.437936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:15.438215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:15.449048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:15.451325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:15.465672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:15.466304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:15.466581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:15.466948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:15.467416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:15.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:15.469241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:15.469241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:15.469241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:15.469412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:15.469483 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 395/784] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:15.473392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:15.477484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:15.480658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 390/785] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:15 2022] GET /identity/v3/services => generated 772 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:15.544626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:15.652430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52ac6a-e062-4f11-af16-a424de3f0d7b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:15.689030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:15.689638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:15.689913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:15.690181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:15.690873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:15.691172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:15.692559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:15.705691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 396/786] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:15.711349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:15.820916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801fda4-c08b-4ac5-84ca-2504fc34533f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:15.823566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:15.861982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:15.862955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:15.863404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:15.863867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:15.872073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:15.873840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:15.885704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:15.887379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:15.888360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:15.890532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 391/787] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:15.896581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:16.031780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7825fd6-98e8-45fb-bdd5-5b7ba85c0ef6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:16.035737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:16.087325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:16.087879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:16.088154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:16.088418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:16.089146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:16.089444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:16.091459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:16.111278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 397/788] 10.222.0.55 () {56 vars in 1135 bytes} [Sat Sep 24 19:32:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:16.120721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:16.264727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afcad1-02c4-4fb2-a725-46c8fc566183 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:16.301331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:16.301961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:16.302238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:16.302508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:16.314570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:16.316061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-00fb94dc-9063-4694-9922-72055c2f4af5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:16.330921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:16.331986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:16.333063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:16.333063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:16.333063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:16.333063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 392/789] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:16.339362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:16.400466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d902894-12d5-4951-aaa7-c45bcb784408 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:16.400813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d902894-12d5-4951-aaa7-c45bcb784408 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:16.401004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d902894-12d5-4951-aaa7-c45bcb784408 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:16.402638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 393/790] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:16.408273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:16.408550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:16.408818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:16.427724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:16.428916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:16.443785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:16.504057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 394/791] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:16.566397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de93b1-1890-49a0-872e-e45371110623 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:16.618286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:16.618810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:16.619132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:16.619402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:16.620103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:16.620400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:16.621784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:16.638220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 398/792] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:16.643519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:16.782572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c80e3-10c9-4b5d-9cc5-ab9032330629 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:16.821807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:16.822440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:16.822715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:16.822983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:16.829979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:16.831447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:16.843384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:16.844309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:16.845221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 395/793] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:16.850223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:16.879878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:16.892562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:16.901505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:16.991804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46fb0-0649-40de-85b1-c469a151241e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.028256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087682-ed20-42ef-92a3-96ba6d949757 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.035724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.036595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.037081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.037584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.038479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:17.039016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:17.040682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbcc11-985b-42b2-baa7-c85fb5098670 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:17.053740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 399/794] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:17.055569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47550b-0ffe-4e2b-9e7f-59ee3513dcdb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.056059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47550b-0ffe-4e2b-9e7f-59ee3513dcdb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.056530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47550b-0ffe-4e2b-9e7f-59ee3513dcdb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.058182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 400/795] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:17.068538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47550b-0ffe-4e2b-9e7f-59ee3513dcdb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.085847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069be86d-30d1-4ea1-b2f5-e9bd60c6c27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.086485 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be86d-30d1-4ea1-b2f5-e9bd60c6c27a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.086768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be86d-30d1-4ea1-b2f5-e9bd60c6c27a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.087093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be86d-30d1-4ea1-b2f5-e9bd60c6c27a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.099985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069be86d-30d1-4ea1-b2f5-e9bd60c6c27a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-069be86d-30d1-4ea1-b2f5-e9bd60c6c27a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:17.105912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:17.106848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:17.107735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:17.108370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 396/796] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:17.112810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.112810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.112810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.129574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:17.130981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:17.159894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:17.199848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47550b-0ffe-4e2b-9e7f-59ee3513dcdb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.205384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:17.214598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 397/797] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:17.216351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.272948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.273992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.275188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.276240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/c72b7d859e954054848d405afa0b466f/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.280576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=c72b7d859e954054848d405afa0b466f, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:17.282837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:17.306341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 401/798] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:32:17 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/c72b7d859e954054848d405afa0b466f/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:32:17.372068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c767f-3003-43f4-8f1f-449809b94dac None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.409252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.410146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.410600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.411020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.411915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:17.412395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:17.414088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6997f-48ff-43b2-b717-683ec2829721 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:17.424841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 398/799] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:17.431156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.547241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109870c-8ce6-436b-aae7-ea9911e00e18 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.547782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109870c-8ce6-436b-aae7-ea9911e00e18 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.548204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109870c-8ce6-436b-aae7-ea9911e00e18 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.549684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 399/800] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:17.550268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b07857-3e2c-4353-ae9f-97abc0d04af0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.556096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.556373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.556640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.582901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:17.584420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:17.593460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:17.612153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.612802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.613084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.613355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.625043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:17.626732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:17.660704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:17.661791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:17.662682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:17.662682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:17.662682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:17.665530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 402/801] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:17.672088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.679779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:17.697563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 400/802] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:17.698836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.802250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5149-6fee-4f38-a449-ab1290403d3d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.843935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.844898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.845421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.845933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.846826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:17.847348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:17.849553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:17.863558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 403/803] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:17.865709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f796-7acb-40a5-b999-833b842e72f1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:17.868569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.925207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:17.954850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:17.955574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:17.955792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:17.956863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:17.956863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:17.958090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:17.972186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:17.973242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:17.974801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 401/804] 10.222.0.55 () {56 vars in 1129 bytes} [Sat Sep 24 19:32:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:17.979249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:17.999658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09249-bfbb-49e5-aa43-71634bc36034 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.044728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.045824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.046305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.046772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.061783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.063248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.080024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:18.081382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:18.082348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 404/805] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:32:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:18.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.144830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424cd97-0bf4-40f7-831f-f847d4823981 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.180593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.181148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.181391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.181660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.182337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:18.182581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.184208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.191629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 402/806] 10.222.0.55 () {56 vars in 1124 bytes} [Sat Sep 24 19:32:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:18.191629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.202060 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b6a4-e097-48c7-96f3-a59594c8b2f6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.204450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.240108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe27145-87d6-4a90-9db3-3e2f30e710db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.240743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe27145-87d6-4a90-9db3-3e2f30e710db None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.241021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe27145-87d6-4a90-9db3-3e2f30e710db None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.241291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe27145-87d6-4a90-9db3-3e2f30e710db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.252024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe27145-87d6-4a90-9db3-3e2f30e710db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-efe27145-87d6-4a90-9db3-3e2f30e710db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:18.257391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:18.258505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 405/807] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:18.264089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d9bd-deb7-43ab-99bb-4a37b80dc115 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.264852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d9bd-deb7-43ab-99bb-4a37b80dc115 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.264852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d9bd-deb7-43ab-99bb-4a37b80dc115 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.264852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 406/808] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:18.264852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58d9bd-deb7-43ab-99bb-4a37b80dc115 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.305162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12cb4b-f37e-44d7-a727-97c7b6241643 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.324080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.343832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.344446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.344724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.344992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.345716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:18.346022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.347472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.365956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 403/809] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:18.366675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.378720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58d9bd-deb7-43ab-99bb-4a37b80dc115 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.416513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.417147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.417425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.417696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.419396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:18.419742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.422470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4466-5fb3-418b-93a7-9f911dbb0c3e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.429039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 407/810] 10.222.0.55 () {56 vars in 1124 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:18.430590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.430936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.431154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.450815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:18.452045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:18.477024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.480699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034076f-b079-4fd7-8a0a-ba41fe2c4923 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.518474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.519076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.519352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.519690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.520346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:18.520646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.522013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.535447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 404/811] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:18.535447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.535819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 408/812] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:18.536734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.570909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.639843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6714b2-7439-4c03-9f80-350ca78666b0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.673354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fb36-2baf-4a47-806f-c901bde320e0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.674255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.674915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.675199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.675520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.718260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.719209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.719514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.719788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.720518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.722569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.752009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.753736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.754844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 409/813] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:18.756094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.779142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.801041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:18.887731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 405/814] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:32:18.889046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:18.896467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2d658-6fc6-48d3-9df9-9f3b13ba26b3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:18.939344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:18.940039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:18.940317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:18.940882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:18.954670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:18.956193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:18.972717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:18.973702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:18.975199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:18.975199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:18.975199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:18.978623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 410/815] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:18.979506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.021497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94a0fa-9c13-44df-99e5-0a8b2d2bb57f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.053337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:19.072572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.072771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.073043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.073305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] PATH_INFO: `/v3/services/network` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.074940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.076892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.080532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:19.090798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:19.091970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 19:32:19.092811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:19.093861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 406/816] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:19.093861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.109368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9086ca-ba56-4930-b9f7-78823b4042b3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.165398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.166033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.166309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.166578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.179128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.180885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:19.197307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:19.198514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:19.200240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:19.200240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:19.200240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:19.200240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 411/817] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:19.200240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.245756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96227165-5bd9-46d1-8727-630341b30033 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.298048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.299754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.300142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.300649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.301797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:19.302150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.304348 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.320196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 407/818] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:19.321431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.329935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39785a-dcc5-435f-a6fb-2b618cbb5285 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.345635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:19.369530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.370205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.370481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.370746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.371953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:19.372255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.373738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.384017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 412/819] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:19.384578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.391526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:19.439157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17564d5b-79b0-4f59-b8bb-c563bb6acd11 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.476888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.477650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.478024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.478384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.479226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:19.479654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.481845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.495218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 408/820] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:19.496873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.497602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92562-5cb4-46ef-aa29-f10f744645e7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.534499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.535531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.536160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.536623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.546092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.547926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:19.561895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:19.562891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:19.563876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:19.566069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 413/821] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:19.566664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.610379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4330-a3ec-481f-a495-662cb964f00f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.647861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:19.652234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.653406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.653850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.654339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.655041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:19.655345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.656865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.667875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 409/822] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:19.667875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.760767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:19.801259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed04191-a57b-4059-8637-ffd9ffa799f9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.836155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44500272-42d6-4b59-921b-b3631e67d746 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:19.849401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.850360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.850645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.850922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.851918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:19.852230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.854198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.874919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 410/823] 10.222.0.55 () {56 vars in 1149 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:19.876133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.881204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:19.881858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.882143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.882411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.890638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:19.892194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-c9d8f3a6-c96b-455e-8e61-4e11951454c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:19.907288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:19.908411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 414/824] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac677b7-2ac7-4be3-ad6e-e9f7410a3c64 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:19.909796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac677b7-2ac7-4be3-ad6e-e9f7410a3c64 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:19.910731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac677b7-2ac7-4be3-ad6e-e9f7410a3c64 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:19.910731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 415/825] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:19.910731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac677b7-2ac7-4be3-ad6e-e9f7410a3c64 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:19.935484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.001928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629cb1c-db93-4666-9da8-21ff1d1a1e3b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.030837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.037204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac677b7-2ac7-4be3-ad6e-e9f7410a3c64 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.039314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.040055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.040329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.040592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.041248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.042610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.075390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 411/826] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:19 2022] POST /identity/v3/endpoints => generated 337 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:20.076675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.114355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.115109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.115773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.116101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.129395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.131102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-cddbbec1-290b-469c-bf17-c66864dc48f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:20.151603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:20.152916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:20.154033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:20.154033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:20.154033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:20.161142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 416/827] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:20.161142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.161424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.161641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.193472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:20.195135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:20.221294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e50b-088b-4f55-8a05-a47cf8f075d8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.224456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.262780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.263909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.264400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.264871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.265906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:20.266540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.268382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.284397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 412/828] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:20.290242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.290365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 417/829] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:20.291993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.390159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.410565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef85ca1-d060-4fa2-abff-cb5a28ce25d9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.419803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba02407-2998-481d-a996-bc1c9b3e44a3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.465588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.466580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.467106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.467640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.469232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.469232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.469232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.469612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.470094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=57f68f991883493dbf78c0bcfd827c1d) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:20.470408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.471311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:20.471869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.472338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.474849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.481758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 418/830] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/role_assignments?user.id=57f68f991883493dbf78c0bcfd827c1d&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:20.490819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 413/831] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:20.496639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.603280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.630326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808d8f-0914-4e1d-afbe-7f8cb63dec53 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.631323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.674440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.675705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.676232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.676741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.687976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.688947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.692830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:20.704237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:20.705146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:20.706040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 419/832] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:20.711310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.744630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdf083-3b20-4b73-8bc0-3cdb6d124d26 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.792145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.792872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.793259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.793580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.820043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:20.828623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:20.829821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:20.830850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 414/833] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:20.836413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.838979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea46ff-f158-4d78-8b01-60a0a2ef2a5a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.879867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:20.880696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:20.881076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:20.881474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:20.882272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:20.882678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:20.884251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:20.899208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 420/834] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:20.905329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:20.930863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:20.962773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a032a3-0107-4156-b831-665fb2a75e1b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:20.969550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:21.003979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.004640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.004919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.005186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.005842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:21.006240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.007796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.013494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b80df-b8e4-4306-9ad1-2da75639c4fb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.023394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 415/835] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:21.024459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.050634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96a3ee-7c63-470f-bbc2-8f579ce60ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.051405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96a3ee-7c63-470f-bbc2-8f579ce60ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.051780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96a3ee-7c63-470f-bbc2-8f579ce60ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.052140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96a3ee-7c63-470f-bbc2-8f579ce60ed2 None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/81fd5bd3dbf743d8b6c6545fa33afdb6/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.078008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96a3ee-7c63-470f-bbc2-8f579ce60ed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=81fd5bd3dbf743d8b6c6545fa33afdb6, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.080976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96a3ee-7c63-470f-bbc2-8f579ce60ed2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.107540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 421/836] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:32:20 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/81fd5bd3dbf743d8b6c6545fa33afdb6/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:32:21.155611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b494392-8f7f-4b69-a26d-8dedc651b70e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.191799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e0961-5166-4bbe-9e26-d6ab481e98b6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.192226 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e0961-5166-4bbe-9e26-d6ab481e98b6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.192560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e0961-5166-4bbe-9e26-d6ab481e98b6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.194423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 422/837] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:21.199751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.200407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.200678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.200941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.201654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.202140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.202607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.211833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:21.216177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.218239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.223905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:21.225429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:21.235390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:21.236464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:21.237381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:21.237381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:21.237381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:21.241604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 416/838] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:21.242680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.320788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 423/839] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:21.359642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcae0-d014-4b0b-8aa4-3363e16890e1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.394625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.395280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.395594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.395862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.396503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:21.396802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.398260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.411303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 417/840] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:21.416962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.463324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:21.534023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443afe1-a508-49a0-9826-3b6a9e5036f9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.571285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.571948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.572227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.572493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.580966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.582501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:21.596532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:21.597568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:21.598606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 424/841] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:21.603786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.617972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.680680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:21.737201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:21.743007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27f12-1973-4902-aece-2b98fbb64bc8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.745774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96aa3-999b-494c-8ea5-355cc6a677e8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.788977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.789993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.790418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.790841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.791832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.792919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.793650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.794047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.794155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.794320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.795050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:21.795376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.797091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:21.804640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:21.805696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:21.806543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:21.810032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 425/842] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:21.819475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.822675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 418/843] 10.222.0.55 () {56 vars in 1135 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:21.825246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:21.938776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5a371-72c7-46a5-98ce-57534de5abed None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.958169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e794e1-c91e-456d-89b0-06cf7e2cdb06 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:21.971491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:21.990709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:21.991949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:21.992656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:21.993106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:21.994466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:21.995051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:21.997317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.001962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:22.002662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.003011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.003355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.009066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 426/844] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:22.011550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:22.048823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:22.097756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:22.099689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.141023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0500a7-cc84-4841-a5c1-bf079e772404 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:22.179993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:22.180854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.181274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.181712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.182485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:22.182940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:22.184578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.190107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 427/845] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:22.226469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 419/846] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:32:22.233911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:22.342108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:22.354348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9869-63c7-4a66-862b-070ae32b0eba None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:22.394696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:22.395670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.396115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.396523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.407827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:22.409459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:22.425050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:22.426232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:22.427341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:22.427341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:22.427341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:22.429061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 428/847] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:22.436312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:22.455956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:22.552713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6ed71-b923-422f-8ab0-896f50caac1c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:22.589502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:22.590283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.590564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.590840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.591510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:22.591816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:22.593200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.605787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 420/848] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:22.611067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:22.716208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9d21a-402d-4b96-9fe9-3a00628d298a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:22.751345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:22.752011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.752270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.752538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.759752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:22.761199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6d5bd8f5-da62-41f4-a2e3-ebabff54b230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:22.773351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:22.774287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:22.775200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 429/849] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:22.782270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:22.863215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:22.897513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32576488-3e1d-47bc-ad5b-eec3a554aa2b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:22.934553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:22.935562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.936048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.936522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.937457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:22.937967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:22.939634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:22.943354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d4291-d347-4e24-8aa8-3c7f8cd65672 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.943890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d4291-d347-4e24-8aa8-3c7f8cd65672 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.944442 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d4291-d347-4e24-8aa8-3c7f8cd65672 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.946431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 430/850] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:22.953779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:22.954345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:22.954853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:22.956547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 421/851] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:22.964256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:22.969891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:22.982159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:22.983373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:23.075016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 431/852] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:23.106438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd303c0-c175-4309-8113-77ebb9c573d7 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:23.137855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:23.145829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:23.146704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.147205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.147713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.148639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=c72b7d859e954054848d405afa0b466f) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:23.149161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:23.150851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f8bfe-e8ee-4491-b794-9a4695915bb7 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:23.158310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 422/853] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:22 2022] GET /identity/v3/role_assignments?user.id=c72b7d859e954054848d405afa0b466f&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:23.355159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:23.420540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94651b3d-aba4-4b29-a54e-6fd50b6b530f None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.421062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94651b3d-aba4-4b29-a54e-6fd50b6b530f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.421062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94651b3d-aba4-4b29-a54e-6fd50b6b530f None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.422630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 423/854] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:23.427710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.427993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.428269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.446990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:23.448217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:23.464347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7515ed-22d1-41a1-8040-af5435a2de12 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:23.504912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:23.505542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.505818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.506092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.506671 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:23.508084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:23.510187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:23.523936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:23.524971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:23.525899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 432/855] 10.222.0.55 () {56 vars in 1132 bytes} [Sat Sep 24 19:32:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:23.527253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:23.539864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 424/856] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:23.643981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ae813-73b0-4ee6-a812-f7009e29fb85 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:23.680699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:23.681306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.681584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.681855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.682540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:23.682841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:23.685210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:23.694524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 433/857] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:23.696529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:23.809242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437977c-5b26-46f2-bd35-547e9136fd3c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:23.811500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:23.835849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:23.848257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:23.849367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.849858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.850371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.851204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:23.851752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:23.854418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:23.860286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 425/858] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:23.908163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:23.938381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c715b-731c-44cb-a58e-088f2a4c1c17 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:23.977864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:23.978637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:23.978922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:23.979193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:23.992218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:23.998660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:23.999723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:24.000985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 434/859] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:32:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:24.010987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.137608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43344f-6673-49d7-a77c-2a15b538b586 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:24.177221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:24.177765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.178047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.178306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.178982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:24.179282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:24.180969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:24.191143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 426/860] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:32:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:24.196452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.304265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0bd0-38bb-42d4-b6c6-4fa2b3d56ba4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:24.329803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:24.341985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:24.342618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.342896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.343168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.353944 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:24.355543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:24.370211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:24.371815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:24.373096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:24.373096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:24.373096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:24.373096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 435/861] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:24.377714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.430978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:24.494295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863db0fc-6d81-41c6-969c-b67117eed5b3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:24.531772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:24.532635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.533057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.533471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.534356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:24.534806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:24.536437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e9a0c-1158-48f8-994c-793f3229ca3c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:24.549918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 427/862] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:24.555931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.565840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17435939-4f7a-4006-b567-936ab698a3e7 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.566417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17435939-4f7a-4006-b567-936ab698a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.566916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17435939-4f7a-4006-b567-936ab698a3e7 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.568924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 428/863] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:24.575499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.576389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.576892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.601028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:24.602871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:24.679332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e7d80-6e01-4b05-9fc5-9bc6a425239e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:24.679984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 429/864] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:24.715222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:24.716104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.716523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.716934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.724990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:24.726614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:24.738669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:24.739813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:24.740970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:24.743035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 436/865] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:32:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:24.749328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.794721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:24.860825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe4f6b-30c7-4735-9e9f-0d2a3c7b04c2 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:24.900242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:24.901236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:24.901709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:24.903052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:24.904148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:24.904651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:24.906280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:24.927885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 430/866] 10.222.0.55 () {56 vars in 1149 bytes} [Sat Sep 24 19:32:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:24.936661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.958886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:24.975677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:25.068579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41b0b9-6a28-47d3-ace9-b2692ec841a6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:25.091169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:25.098095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cfb52-8d2a-4b80-9094-43ac2375da03 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:25.118269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a4855-b573-44c2-9db2-e59179587a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:25.118965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4855-b573-44c2-9db2-e59179587a2e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.119278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4855-b573-44c2-9db2-e59179587a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.119643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4855-b573-44c2-9db2-e59179587a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.156597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:25.157284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.157567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.157843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.159602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:25.161158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:25.194446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 431/867] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:24 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:32:25.212212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a4855-b573-44c2-9db2-e59179587a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:25.213748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a4855-b573-44c2-9db2-e59179587a2e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:25.255904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:25.329673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 437/868] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:32:25.336652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:25.465645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccbaee-35a8-4d0e-aa8f-d55b16458c3a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:25.508866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:25.510316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.511005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.511790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.529061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:25.530913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:25.541844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706105e6-0e7e-43ee-8707-7a2e73a66ca7 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.542128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706105e6-0e7e-43ee-8707-7a2e73a66ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.542399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706105e6-0e7e-43ee-8707-7a2e73a66ca7 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.548653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 438/869] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:25.550120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.550406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.550683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:25.552409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:25.553346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:25.554276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:25.554276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:25.554276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:25.556434 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 432/870] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:25.562775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:25.573631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:25.575517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:25.575776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:25.655566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 439/871] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:25 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:25.680325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40f92-7223-466d-ba19-6a6678f00a01 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:25.716129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:25.716735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.717009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.717277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.717916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:25.718216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:25.719605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:25.733764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 433/872] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:25.739110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:25.807581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:25.844773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e374af1-8f94-44ab-bfdd-1d0fb0d0c546 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:25.879454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:25.880089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:25.880411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:25.880681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:25.887529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:25.888949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:25.900038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:25.900896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:25.902614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 440/873] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:25.908663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:25.943999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:25.990396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:26.041648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b3cf-af48-4f48-b59b-a14e75800029 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:26.065792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12791aa-ec6a-4f9b-871d-87eca7167f42 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:26.088070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:26.089101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.089738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.090861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.091945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:26.092545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:26.094367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:26.094488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:26.110486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:26.111222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.111608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.111949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.112629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:26.113383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 434/874] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:26.119000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:26.119872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:26.130258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:26.131360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:26.132247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:26.136994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 441/875] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:26.138177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:26.277153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e16e-9489-46b8-b637-da8ff0d952b1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:26.281416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493db11-ea87-40b3-9d66-6eb7da947b2b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:26.305520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:26.322827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:26.323366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.323667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.323940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.325589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=81fd5bd3dbf743d8b6c6545fa33afdb6) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:26.325906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:26.327370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:26.335734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:26.336654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.337130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.337595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.338425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:26.338922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:26.339864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 435/876] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:26 2022] GET /identity/v3/role_assignments?user.id=81fd5bd3dbf743d8b6c6545fa33afdb6&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:26.341026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170167-2391-4f91-a285-c6de967976d8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:26.347269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 442/877] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:26.352618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:26.363852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95680e-e35e-471c-b5c4-ee471c566483 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.364101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95680e-e35e-471c-b5c4-ee471c566483 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.364938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95680e-e35e-471c-b5c4-ee471c566483 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.370264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 443/878] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:26.372690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a777c-1493-4766-b047-ef124ed1004e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.372993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a777c-1493-4766-b047-ef124ed1004e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.373270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a777c-1493-4766-b047-ef124ed1004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.395032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-187a777c-1493-4766-b047-ef124ed1004e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:26.396432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-187a777c-1493-4766-b047-ef124ed1004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:26.451934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:26.482863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 444/879] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:26 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:26.494851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f14dea-b9e4-4365-9f9a-73f602a9d178 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:26.534436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:26.535048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.535323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.535619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.536248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:26.536546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:26.537906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88eef5-1c7f-4589-a227-1479d52ab1fb None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:26.542858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 436/880] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:26.757149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a777c-1493-4766-b047-ef124ed1004e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:26.863231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a777c-1493-4766-b047-ef124ed1004e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:26.900087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba2313-07c0-4a0d-8ba5-24d45ca0c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:26.901028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba2313-07c0-4a0d-8ba5-24d45ca0c09d None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:26.901485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba2313-07c0-4a0d-8ba5-24d45ca0c09d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:26.901943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba2313-07c0-4a0d-8ba5-24d45ca0c09d None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:26.903372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba2313-07c0-4a0d-8ba5-24d45ca0c09d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:26.904965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba2313-07c0-4a0d-8ba5-24d45ca0c09d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:26.926396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 445/881] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:26 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:27.512130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2846163-29e0-4568-b8c3-872a30abaacf None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:27.512668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2846163-29e0-4568-b8c3-872a30abaacf None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:27.512668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2846163-29e0-4568-b8c3-872a30abaacf None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:27.514342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 437/882] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:27.519728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:27.520146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:27.520315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:27.538427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:27.539668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:27.578261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:27.618297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 446/883] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:27.893872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2846163-29e0-4568-b8c3-872a30abaacf None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:27.953420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:28.008428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2846163-29e0-4568-b8c3-872a30abaacf None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:28.042893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:28.043767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.044028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.044311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.044884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:28.046267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:28.057533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:28.059099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:28.060070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 438/884] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:28.067859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:28.171666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c230544-587e-4ee2-9bfa-12c78bbd5be0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:28.208203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:28.208862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.209137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.209401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.210084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:28.210384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:28.211763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:28.217197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 447/885] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:28.222707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:28.337783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30db0a-12e8-44d5-b8df-8296d3929921 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:28.357451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:28.377704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:28.378319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.378598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.378868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.379529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:28.379833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:28.381211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:28.387209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 439/886] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:28.393284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:28.426767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:28.504904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f080499-dd54-49d8-bdac-b8327024522d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:28.540824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:28.541468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.541686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.541990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.543293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] RBAC: Adding query filter params (interface=public, service_id=ca82e048c8dd4d6e9eef6fe1de3405a0, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:28.543758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:28.545415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a19bf3-b06e-497c-87bd-394397297f7a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:28.551529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 448/887] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:28 2022] GET /identity/v3/endpoints?service_id=ca82e048c8dd4d6e9eef6fe1de3405a0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:28.557196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:28.671232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc27ce-776e-4474-a7b8-5cb1887429cc None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:28.702724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2938234-6b48-4f9f-aae3-ac3721850ea6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.702911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2938234-6b48-4f9f-aae3-ac3721850ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.703193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2938234-6b48-4f9f-aae3-ac3721850ea6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.704849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 449/888] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:28.708276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467630c5-c137-411b-85cc-34361e78fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:28.709006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467630c5-c137-411b-85cc-34361e78fb6a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.709283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467630c5-c137-411b-85cc-34361e78fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.709551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467630c5-c137-411b-85cc-34361e78fb6a None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.710166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467630c5-c137-411b-85cc-34361e78fb6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:28.710273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9ec51-5a42-4c99-b9b0-91e594da5d9b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.710519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9ec51-5a42-4c99-b9b0-91e594da5d9b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.710800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9ec51-5a42-4c99-b9b0-91e594da5d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.711586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467630c5-c137-411b-85cc-34361e78fb6a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:28.720061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 440/889] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:28 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:32:28.733117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-42d9ec51-5a42-4c99-b9b0-91e594da5d9b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:28.734381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-42d9ec51-5a42-4c99-b9b0-91e594da5d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:28.841164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 450/890] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:28.904092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b052a-7dd4-44fa-84b7-d3a255dea0c8 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.904564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b052a-7dd4-44fa-84b7-d3a255dea0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.905009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b052a-7dd4-44fa-84b7-d3a255dea0c8 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.907443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 441/891] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:28.913446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651e3c1-0fa7-446a-adea-1e19c6f2fa8c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:28.913752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651e3c1-0fa7-446a-adea-1e19c6f2fa8c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:28.914058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651e3c1-0fa7-446a-adea-1e19c6f2fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:28.945021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d651e3c1-0fa7-446a-adea-1e19c6f2fa8c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:28.947297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d651e3c1-0fa7-446a-adea-1e19c6f2fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:28.977258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:29.033541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 451/892] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:29.122721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b052a-7dd4-44fa-84b7-d3a255dea0c8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.179292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:29.238818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b052a-7dd4-44fa-84b7-d3a255dea0c8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:29.254560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d52ff-311b-4f54-9736-1a02a92b1ced None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.255170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d52ff-311b-4f54-9736-1a02a92b1ced None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.255735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d52ff-311b-4f54-9736-1a02a92b1ced None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.257615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 452/893] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:29.263445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.264103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.264569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.280765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:29.280765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.280765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.280765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.287693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:29.289202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:29.291162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:29.298155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:29.299108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:29.300005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 442/894] 10.222.0.55 () {56 vars in 1120 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:29.305911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.373218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 453/895] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:29.374177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.419438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff97ec-452f-4468-a7e6-c1121ed023b1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:29.427334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:29.458451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:29.459224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.459678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.460049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.460817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:29.461225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:29.463255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:29.474456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:29.479672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 443/896] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:29.484601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.497459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb0602-e5af-4e9e-849d-154073056a2c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:29.538747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:29.539854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.540766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.541262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.543582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:29.545444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:29.566295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 454/897] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:29 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:29.619950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2cc64-0037-4fb6-8e85-25cedecc0980 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:29.655296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.671018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:29.671701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.671982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.672262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.685910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:29.687282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:29.708235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:29.709502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:29.710584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:29.710584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:29.710584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:29.715615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 444/898] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:29.717100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.738935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:29.809889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5350f3-8531-440e-a0b6-eb04a6a5b610 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:29.837903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00765e43-9d2c-44bc-b0ef-485228eec398 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:29.858174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:29.859554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.859970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.860379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.860976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:29.863068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:29.873895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:29.874615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:29.874982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:29.875227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:29.876159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:29.877072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:29.877805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:29.877805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:29.877805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:29.878094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 455/899] 10.222.0.55 () {56 vars in 1132 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:29.878166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:29.885844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.898202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 445/900] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:29.904103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:29.906528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:29.987800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:30.014561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4718-2d5f-41ec-964f-66a1a6a6bbd5 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.023648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb89d8-aa50-425c-9c7c-ed47f40c321a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.052330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.053252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.053677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.054090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.064003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.064919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.065830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.066015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.066297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.066571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.067204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:30.067539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.069189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.078004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 456/901] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:30.079879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:30.080832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:30.081770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:30.083883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 446/902] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:32:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:30.088116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.090178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.188832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:30.218514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528d5dc-aafc-4f43-a2c2-7ea8097bf2be None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.222917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e54ccb-e96c-49d3-b985-d228e32be46c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.257463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.258476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.259233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.259783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.260749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:30.261256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.264019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.265414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.266099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.266596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.266911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.267631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:30.267979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.268248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:30.270510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.279583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 457/903] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:30.290083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.299665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 447/904] 10.222.0.55 () {56 vars in 1149 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:30.301368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.416093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12c51f-b380-41fa-892c-377edd2761be None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.423138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ba1ae-ebf2-4bc2-bbd4-90d65496c434 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.457621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.458615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.459277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.459944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.467173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.467818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.468098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.468359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.469065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] RBAC: Adding query filter params (interface=public, service_id=974d98a58a8b452f8f53a49ebbe5498a, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:30.469373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.470754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.479582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 458/905] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/endpoints?service_id=974d98a58a8b452f8f53a49ebbe5498a&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:30.484068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.488231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:30.551626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:30.563644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.565630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.635948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181ad3-b798-4a48-9d77-d1b5a46af57b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.675856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.676701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.677082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.677455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.678261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.679995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.682387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 448/906] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 19:32:30.689182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.695503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 459/907] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/services => generated 1497 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:32:30.804732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:30.814843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b01d-c2da-4bbd-a4fb-0655d95900d8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:30.860666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:30.861803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:30.862309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:30.862824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:30.880841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:30.883271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-5063dd89-c0ca-43bb-b325-36d4a1837a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:30.907334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:30.908484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:30.909781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:30.909781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:30.909781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:30.909781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 449/908] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:30.916340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:30.999835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c11d7-a8ce-4103-ad42-f83488771dac None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.000430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c11d7-a8ce-4103-ad42-f83488771dac None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.000873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c11d7-a8ce-4103-ad42-f83488771dac None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.003490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 450/909] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:31.004071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:31.015547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.016572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.017204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.040534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:31.042121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:31.046994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eea6c-0d96-400a-b0a5-a49713a517e5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.093806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.094580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.094948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.095315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.096129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:31.096521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.098081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.115582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 460/910] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:31.120620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:31.151461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 451/911] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:31.237621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac882317-518d-4881-9e50-62e0d2d292f4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.270470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:31.276121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.276753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.277661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.278025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.285710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.287204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:31.299232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:31.300262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:31.301301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 461/912] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:31.308654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:31.391868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:31.418616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae2106-bc58-4112-ae2e-351ae3447e1b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.428354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:31.464945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.465698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.465976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.466269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.467626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:31.467939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.469944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.486576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 452/913] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:31.492144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:31.569177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826547f-3920-41fa-9a41-908f8e263939 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.611211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbbddb-2506-4246-853f-798c7ec00e33 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.613675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.614548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.615000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.615467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.616236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.617833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:31.628942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:31.629961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 19:32:31.630809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:31.633261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 462/914] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:31.640321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:31.655025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.655987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.656486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.656985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.657876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] RBAC: Adding query filter params (role.id=c00a7c03d65e424381c3a31a685c3016, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=81fd5bd3dbf743d8b6c6545fa33afdb6) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:31.658383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.660191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.663412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:31.675619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 453/915] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:31 2022] GET /identity/v3/role_assignments?user.id=81fd5bd3dbf743d8b6c6545fa33afdb6&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=c00a7c03d65e424381c3a31a685c3016 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:31.702297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:31.765301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb41f3-5a46-4fba-ad68-a34af64d26c3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.809564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.810196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.810476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.810747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.811410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:31.811748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.813170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b668-f4ac-42e2-9575-6aefd686410c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.819354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 463/916] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:31.825126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:31.910075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722e92-f585-4d2a-ac9c-b2319085133f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.910620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722e92-f585-4d2a-ac9c-b2319085133f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.911057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722e92-f585-4d2a-ac9c-b2319085133f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.912810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 464/917] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:31.920267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.920967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.921408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.944534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347d209-a084-43d2-9f5c-93402bfc03d1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:31.945139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:31.946647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:31.984577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:31.985354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:31.985727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:31.986142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:31.986802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:31.987230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:31.989047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:31.994631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 454/918] 10.222.0.55 () {56 vars in 1114 bytes} [Sat Sep 24 19:32:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:32.004177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:32.037447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 465/919] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:32.081430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:32.125065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b49ee-a8f8-4c54-802d-9fd118c28479 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:32.161283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:32.162198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:32.162681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:32.163126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:32.163993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:32.165553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:32.190600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 455/920] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:32:32.347241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:32.398439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:32.468345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ad424-66bc-464e-8382-95a4e544217e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:32.512397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:32.513588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:32.514096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:32.514580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:32.515360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:32.517108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:32.526869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:32.528455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:32.529313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:32.532263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 466/921] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:32.538455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:32.651119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34db1c3-3b8e-4f5d-a3b5-bc5deb31a094 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:32.678662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:32.694994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:32.695666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:32.696101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:32.696376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:32.697025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:32.697327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:32.698711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:32.704181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 456/922] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:32.709849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:32.826026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64966ee7-6bb7-40fd-9a34-2d4b959c2928 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:32.828770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:32.864153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:32.865100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:32.865609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:32.866078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:32.866997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:32.867533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:32.869098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:32.875871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 467/923] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:32.882174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:33.000026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a08fe-744b-4dc5-954d-c4725c6258bd None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:33.035166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:33.035821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.036101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.036375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.037079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] RBAC: Adding query filter params (interface=public, service_id=5fd3c2ac23b447848b6a016fdf639fcf, region_id=RegionOne) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:33.037389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:33.038779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa8ec5-d9d5-4961-93d7-f710114fc1f3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:33.048014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 457/924] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:32 2022] GET /identity/v3/endpoints?service_id=5fd3c2ac23b447848b6a016fdf639fcf&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:33.053949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:33.071896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddc128-563e-40fc-b7a4-0a6e502d5448 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.072191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddc128-563e-40fc-b7a4-0a6e502d5448 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.072456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddc128-563e-40fc-b7a4-0a6e502d5448 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.074232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 458/925] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:33.079659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.079950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.080226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.099645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:33.100853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:33.155290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:33.183489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f983f-2d68-4477-9aa7-6b92207cf377 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:33.191660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 459/926] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:33.220230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:33.221162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.221666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.222147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.222970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:33.224551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:33.232108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 468/927] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:33 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:32:33.456884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:33.567237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeb4bc-ddd4-4d86-9e9b-198b930aa607 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:33.603890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:33.604537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.604818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.605661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.606295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:33.607666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:33.617092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:33.618044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 19:32:33.618934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:33.623159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 460/928] 10.222.0.55 () {56 vars in 1132 bytes} [Sat Sep 24 19:32:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:33.625410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:33.664241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:33.739118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392736c-dc13-4154-934a-020c22ae5b95 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:33.776738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:33.777646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.777945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.778181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.778809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:33.779108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:33.780507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:33.785721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 469/929] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:33.793406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:33.841739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:33.907368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da24148-790c-4bd7-9364-127b2360927c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:33.945460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:33.946484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:33.946987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:33.947502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:33.948326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:33.948861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:33.950439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f923-7cbc-4068-88e7-c5a931cf2d40 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:33.956294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 461/930] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:33.965163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:33.999574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368f9c-c5f4-4d4c-b031-5ca98ae5bad9 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.000196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368f9c-c5f4-4d4c-b031-5ca98ae5bad9 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.000572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368f9c-c5f4-4d4c-b031-5ca98ae5bad9 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.002260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 462/931] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:34.008670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.009097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.009469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.028562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:34.029911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:34.078055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:34.086192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ecec-0c43-49d3-9bce-d33b17c36611 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:34.119563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 463/932] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:34.127349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880e551-1cb6-47a2-b29e-a9022f74fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:34.128233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880e551-1cb6-47a2-b29e-a9022f74fbb3 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.128663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880e551-1cb6-47a2-b29e-a9022f74fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.129084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880e551-1cb6-47a2-b29e-a9022f74fbb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.129975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880e551-1cb6-47a2-b29e-a9022f74fbb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:34.131519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880e551-1cb6-47a2-b29e-a9022f74fbb3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:34.159589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 470/933] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:34.395716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:34.432210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:34.464068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101c0bb-ed0a-4aa7-b2d4-43140025c2d9 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.464441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101c0bb-ed0a-4aa7-b2d4-43140025c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.465367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101c0bb-ed0a-4aa7-b2d4-43140025c2d9 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.466974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 471/934] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:34.472312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.472590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.472860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.493572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:34.494759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:34.503835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ebf01-fab0-45c4-bd31-903740cca2c1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:34.547507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:34.548447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.548922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.549353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.564299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:34.571326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:34.572302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:34.573153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:34.581135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 464/935] 10.222.0.55 () {56 vars in 1120 bytes} [Sat Sep 24 19:32:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:34.582271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:34.591521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 472/936] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:34.682365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:34.692492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cce64-4967-4cba-8067-94c198ca0179 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:34.733621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:34.734232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.734506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.734774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.735459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:34.735759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:34.737134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:34.746652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 465/937] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:32:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:34.756530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:34.843417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:34.870451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929673-d1df-4c1b-a93e-ca3589ee6df4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:34.875956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:34.909954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:34.910589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:34.910866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:34.911132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:34.925278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:34.926740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:34.946737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:34.947753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:34.948636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:34.948636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:34.948636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:34.948636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 473/938] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:34.955274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:34.987617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418e5b3-34d8-4a86-afe0-439d8874fe33 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.030445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.031188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.031552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.031890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.033182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.035097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.039091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING keystone.server.flask.common [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 19:32:35.066786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.sql.core [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 19:32:35.066786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 19:32:35.066786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [parameters: {'id': 'fc39eaf659c94aeda3b70b73126a4cc6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 19:32:35.066786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81801) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fc39eaf659c94aeda3b70b73126a4cc6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fc39eaf659c94aeda3b70b73126a4cc6. Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application conn.query(q) Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 19:32:35.080276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result.read() Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 19:32:35.081203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application next(self.gen) Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application yield resource Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application next(self.gen) Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise self.value Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application session.commit() Sep 24 19:32:35.082231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.session.flush() Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application compat.raise_( Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 19:32:35.083145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 19:32:35.084091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application conn.query(q) Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result.read() Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 19:32:35.084903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [parameters: {'id': 'fc39eaf659c94aeda3b70b73126a4cc6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.093071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:35.094939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fc39eaf659c94aeda3b70b73126a4cc6. Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 466/939] 10.222.0.55 () {60 vars in 1140 bytes} [Sat Sep 24 19:32:34 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:35.097117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.103595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1f6f-3f7e-4273-bad4-107958bfddd1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.107855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:35.145583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.146601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.147100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.147627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.148669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:35.149188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.150914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.154687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:35.172348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 474/940] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:35.177863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.224695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddec4ae-5dd9-4835-a98c-cb903626e31e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.261495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.262006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.262287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.262559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.269610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.271255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:35.283785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:35.284733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:35.285651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.289743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 467/941] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:35.290616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.303615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c8f8-5841-4a48-93c1-3cb66bb565f6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.346331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.347326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.347873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.348461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.358100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.360151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.368786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:35.376384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:35.377366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:35.378333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:35.385727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 475/942] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:35.386413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.411552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead1978-9356-480d-b6a7-88cba0ccf280 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.448449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.449098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.449374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.449641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.450321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:35.450619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.452146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d138da-5f6e-4c75-a4ce-a369e89a81f0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.461540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:35.477769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 468/943] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:35.522434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ff5d6-ef83-4511-b38a-98360c304eb1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.563360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.564398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.564876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.565355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.566308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:35.566821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.568506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c0f35-49ba-4a37-9aa2-a5ccf1063a08 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.589956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c50045-53a4-4f39-b01b-fca9ab5ff2a2 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.590289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c50045-53a4-4f39-b01b-fca9ab5ff2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.590510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c50045-53a4-4f39-b01b-fca9ab5ff2a2 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.591182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 476/944] 10.222.0.55 () {56 vars in 1149 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:35.595468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 469/945] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:35.597387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c50045-53a4-4f39-b01b-fca9ab5ff2a2 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.598110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.598663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.599133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.621534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:35.623660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:35.713378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 477/946] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:35.720247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c50045-53a4-4f39-b01b-fca9ab5ff2a2 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.757232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:35.765585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.766645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.767138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.767698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.779078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.780577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:35.796475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:35.797497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:35.798411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:35.798411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:35.798411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:35.798411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 470/947] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:35.803558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.850093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:35.909164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e471-5d59-4253-a755-6b976b9f0fc2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:35.945397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:35.946011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:35.946285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:35.946549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:35.947280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:35.947610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:35.949113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:35.961676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 478/948] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:35.967572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:35.994961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.098417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d57e11-d5bb-4dc7-9fd3-a3169f523a95 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.125008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a7635-af9e-4b7a-861d-45b98107c39d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.125008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:36.151088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.152540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.153093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.153676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.168646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.169346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.169731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.170101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.170560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.170879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.172387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.172508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:36.184216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:36.185513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:36.191646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 479/949] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:36.192498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:36.193345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:36.194233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:36.195702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 471/950] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:36.196120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.197098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.247809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:36.305854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e33dcb-23f0-4aee-9951-8af04a693834 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.317845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e4cb-aaa7-4b13-bd19-05059a4e3762 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.344791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.345818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.346580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.347067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.348060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:36.348589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.350352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.363738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.364378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.364660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.364935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.365921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:36.366232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.367773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.368251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 472/951] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:36.376153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.385398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 480/952] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:36.386840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.437039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:36.516039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c8a46-9939-460c-a52d-a9d3cfbb790c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.518903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e890-cde0-491e-810d-b2a852b15a11 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.520948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:36.552431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.553053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.553328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.553592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.554240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:36.554792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.556218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a690-6240-44dd-9ade-ab2fba8b4f3b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.562691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 481/953] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:36.562691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d03566-3c23-4252-9df6-34a372e1e881 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.562691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d03566-3c23-4252-9df6-34a372e1e881 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.562691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d03566-3c23-4252-9df6-34a372e1e881 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.562691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 482/954] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:36.571717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.571937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.572205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.573438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.574316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.574744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.575167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/81fd5bd3dbf743d8b6c6545fa33afdb6/roles/c00a7c03d65e424381c3a31a685c3016` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.577579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=81fd5bd3dbf743d8b6c6545fa33afdb6, role_id=c00a7c03d65e424381c3a31a685c3016)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.579174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.590772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:36.592017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:36.603562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 473/955] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:32:36 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/81fd5bd3dbf743d8b6c6545fa33afdb6/roles/c00a7c03d65e424381c3a31a685c3016 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:32:36.603562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.670527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 483/956] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:36.746840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4f20-6ba6-4e3b-9825-4a7f954a4713 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:36.786697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:36.787610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:36.788067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:36.788484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:36.789331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:36.790860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:36.817064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 474/957] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:36.955435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:36.984395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:37.068651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b6f42-7dc4-48a4-9a49-864b6d2647e1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:37.112089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:37.112723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:37.113002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:37.113273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:37.113860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:37.115223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:37.124982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:37.126125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:37.127011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 484/958] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:37.132214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:37.153871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:37.241662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac682d-1320-44e8-83d3-db19c0b93403 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:37.278181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:37.278800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:37.279079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:37.279350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:37.280008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:37.280310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:37.281685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9036145-fbc3-4cf4-b475-ca8e6ecc294f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:37.287003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 475/959] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:37.292667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:37.385868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:37.403938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825750c-2654-4b41-a713-7fe5ac46f74f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:37.444985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:37.445841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:37.446266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:37.446681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:37.447510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:37.447956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:37.449587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ab93b-d772-4d9f-8203-23c0c5d5c0d0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:37.459129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 485/960] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:37.855982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae809c7f-6d79-4043-85cd-1b51a317ebe6 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:37.856263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae809c7f-6d79-4043-85cd-1b51a317ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:37.856532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae809c7f-6d79-4043-85cd-1b51a317ebe6 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:37.858114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 476/961] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:37.863578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:37.863866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:37.864137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:37.882034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:37.883204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:37.951926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 486/962] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:38.223405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae809c7f-6d79-4043-85cd-1b51a317ebe6 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:38.315556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:38.335618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae809c7f-6d79-4043-85cd-1b51a317ebe6 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:38.371307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:38.372030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:38.372311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:38.372585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:38.373241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:38.374737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:38.390180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.sql.core [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 19:32:38.390180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 19:32:38.390180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [parameters: {'user_id': '7a937de17b7d4fd89305cfdd58a1a111', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 19:32:38.390180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81801) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application conn.query(q) Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result.read() Sep 24 19:32:38.398026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 19:32:38.399009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application next(self.gen) Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application yield resource Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application next(self.gen) Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise self.value Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application session.commit() Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 19:32:38.400291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.session.flush() Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application compat.raise_( Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 19:32:38.401366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 19:32:38.402802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application conn.query(q) Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application result.read() Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 19:32:38.403810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [parameters: {'user_id': '7a937de17b7d4fd89305cfdd58a1a111', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:38.406221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 24 19:32:38.408132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:38.409967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 477/963] 10.222.0.55 () {60 vars in 1131 bytes} [Sat Sep 24 19:32:38 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:38.410470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:38.496618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:38.519408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279c8c3-b1b2-48d2-8ce3-18f3a8293e9e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:38.556460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:38.557143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:38.557423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:38.557697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:38.566142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:38.567633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-e6198899-e5da-4f08-8d63-5f28ef533a95 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:38.581345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:38.582687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:38.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 487/964] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:38.592753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:38.700735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3274e-3b0d-4341-8adb-0f295c58ee2b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:38.737647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:38.738268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:38.738606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:38.738882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:38.739606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:38.739911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:38.741561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c454542-3f86-4df7-a970-ab7ef13f3e49 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:38.785651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 478/965] 10.222.0.55 () {56 vars in 1101 bytes} [Sat Sep 24 19:32:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:38.909537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdad6d-1708-4f51-bde6-fc2ddee0248b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:38.909815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdad6d-1708-4f51-bde6-fc2ddee0248b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:38.910088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdad6d-1708-4f51-bde6-fc2ddee0248b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:38.911783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 488/966] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:38.917729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d83fe-80c5-4bca-a444-ecdc7c5dc888 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:38.918011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d83fe-80c5-4bca-a444-ecdc7c5dc888 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:38.918280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d83fe-80c5-4bca-a444-ecdc7c5dc888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:38.925285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:38.941103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ba9d83fe-80c5-4bca-a444-ecdc7c5dc888 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:38.942697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ba9d83fe-80c5-4bca-a444-ecdc7c5dc888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:39.019412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 479/967] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:39.151688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8157-e174-4d2f-bffe-094a4ef81993 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.152845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8157-e174-4d2f-bffe-094a4ef81993 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.153356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8157-e174-4d2f-bffe-094a4ef81993 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.154914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 489/968] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:39.160760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.161047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.161314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.179999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:39.181167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:39.248980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 480/969] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:39.303544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8157-e174-4d2f-bffe-094a4ef81993 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.407678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8157-e174-4d2f-bffe-094a4ef81993 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:39.443132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:39.443802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.444048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.444298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.451074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:39.456412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:39.461260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:39.462204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:39.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 490/970] 10.222.0.55 () {56 vars in 1120 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:39.468440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.515040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.531947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:39.573760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62133bdd-a4ae-4d6e-945d-4e4f9c9876d6 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:39.610571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:39.611183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.611482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.611756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.612530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:39.612862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:39.614314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:39.618971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8977e7-1225-479b-826f-f5a6608e37fb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:39.623788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 481/971] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:39.629286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.657280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:39.658024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.658252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.658508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.659161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:39.660613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:39.672557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:39.673891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:39.674739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 491/972] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:39.680202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.744856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ceb484-e423-47e2-b3d6-f2ed93e2ecaf None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:39.752073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:39.770156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:39.782907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:39.783574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.784304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.784582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.795906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:39.796432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c81a9-78f7-4788-a99c-1fb59ddd12f4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:39.797368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-a5ff6d09-96bc-4b2b-aea0-aa23347e3ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:39.814017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:39.815130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 482/973] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f81841-d150-4c72-8370-bddfbc48be92 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f81841-d150-4c72-8370-bddfbc48be92 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f81841-d150-4c72-8370-bddfbc48be92 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.820885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 483/974] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:39.822431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f81841-d150-4c72-8370-bddfbc48be92 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.831776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:39.832380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.832653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.832917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.833541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:39.833837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:39.835183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6496f2a-6ef3-4d99-a498-e97e2fd8a9c4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:39.839920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 492/975] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:39.841113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.841395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.841683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.859946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:39.861267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:39.924844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f81841-d150-4c72-8370-bddfbc48be92 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:39.931567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 493/976] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:39.932398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:39.975292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:39.975961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:39.983255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:39.983255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:39.983255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:39.983255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:39.983255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:39.996471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 484/977] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:40.001753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.002485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.038328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.052688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0613eb-3a4a-4ec0-a269-24bfcd6509a4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.097142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.097766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.098046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.098317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.098958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:40.099261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.100676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.107194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 494/978] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:40.112909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.115212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e3d3-da74-4e3a-b86e-984b59a787ad None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.151101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.151886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.152233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.152502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.160813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.162256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:40.175237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:40.181718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:40.182697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 485/979] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:40.184509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.225397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18f0c6-c8fb-4f65-8116-0a73341166e6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.245795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.260412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.261560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.262037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.262514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.263391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] RBAC: Adding query filter params (interface=admin, service_id=5fd3c2ac23b447848b6a016fdf639fcf, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:40.263746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.265166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.270263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 495/980] 10.222.0.55 () {56 vars in 1249 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/endpoints?service_id=5fd3c2ac23b447848b6a016fdf639fcf&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:40.271033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.281928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.288893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f18d19-5d19-43fb-8a6b-858d7018707c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.324241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.324897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.325178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.325805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.326175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:40.326480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.327995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.345930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 486/981] 10.222.0.55 () {56 vars in 1149 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:40.346764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.373230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5de0a-640c-4917-ac4b-b3997fe831f8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.406580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.407182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.407478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.407743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.408389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.409740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.428342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 496/982] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:40 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:40.429087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.458860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc12a3-f4a9-45f9-9cf4-e31769acb1ab None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.485793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.508433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.509065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.509352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.509619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.510240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.512004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.519236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 487/983] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:32:40.569098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a458d8-6a02-4ffb-8c9b-dfd7b95c6983 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.611135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.611790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.612063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.612330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.690538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.691958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.743550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.804200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 497/984] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:32:40.810394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:40.826781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:40.916213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6b2e-18e9-48d3-9e9a-f8634e82bdd2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:40.954772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:40.955334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:40.955656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:40.955924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:40.967651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:40.969112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:40.984301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:40.985307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:40.986223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:40.986223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:40.986223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:40.987091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 488/985] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:40.991039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:41.119145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66588a08-a18f-41b3-890f-059998d80572 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:41.155499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:41.156271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:41.156641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:41.157005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:41.157741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:41.158137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:41.159725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:41.172151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 498/986] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:41.177571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:41.248187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:41.286622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c6f6b-5dbf-41df-8fbf-44c6e7d25707 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:41.321631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:41.322406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:41.322779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:41.323140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:41.330843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:41.332453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:41.344229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:41.345328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:41.346436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:41.347988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 489/987] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:41.353937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:41.358017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:41.463907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5bfb-12f7-402d-911f-19134cdecbf7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:41.498842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:41.499633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:41.499633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:41.499902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:41.500592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:41.500893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:41.502424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca78fc-f067-4625-84b4-962f3aa308ad None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:41.515549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 499/988] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:41.521459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:41.623901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618245a-97bc-44ca-9c9d-4b5d55ad62f9 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:41.650072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:41.660222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:41.660834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:41.661108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:41.661372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:41.662083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] RBAC: Adding query filter params (role.id=c00a7c03d65e424381c3a31a685c3016, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=81fd5bd3dbf743d8b6c6545fa33afdb6) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:41.662383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:41.663838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c0e7-f870-41d0-8563-8e42da0b384e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:41.675570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 490/989] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:41 2022] GET /identity/v3/role_assignments?user.id=81fd5bd3dbf743d8b6c6545fa33afdb6&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=c00a7c03d65e424381c3a31a685c3016 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:42.680849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32826071-22c5-4428-a735-08c246ae40e4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:42.681493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32826071-22c5-4428-a735-08c246ae40e4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:42.681493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32826071-22c5-4428-a735-08c246ae40e4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:42.682739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 500/990] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:42.688599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:42.689087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:42.689175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:42.708574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:42.709788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:42.743361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7088f-5283-4cc9-8341-910b6adf45a3 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:42.743735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7088f-5283-4cc9-8341-910b6adf45a3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:42.743934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7088f-5283-4cc9-8341-910b6adf45a3 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:42.745277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 501/991] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:42.751012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:42.751292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:42.751595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:42.769521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:42.770727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:42.786276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 491/992] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:42.835574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:42.844871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 502/993] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:43.064856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.115760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.180256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d82c-de13-4bcb-989f-f4cf7e5b8c24 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.202155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:43.217883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.218521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.218798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.219064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.219744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.221178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.228580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d8273-129b-45b1-a57b-7d7c9f9ca6a4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:43.230151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:43.231347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:43.232321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:43.233816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 492/994] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:43.237256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.268812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.269712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.270020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.270673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.271585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.272973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:43.283643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:43.284699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:43.285673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 503/995] 10.222.0.55 () {56 vars in 1126 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:43.290522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.363111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5e99e-11c0-4790-83f0-7069266f95d0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.367774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:43.400992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.401670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.402458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.402739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.403904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:43.404217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.405607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.415826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 493/996] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:43.416885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.420195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff54f2-39e9-4b10-a833-e96e2de670d4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.457510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.458121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.458396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.458657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.459913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:43.460213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.462013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.471186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:43.474098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 504/997] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:43.475032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.531265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c063bf-4f5f-4f2e-99c1-82000edc9d2e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.570647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.571261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.571574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.571845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.572615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:43.572919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.575105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.585254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 494/998] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:43.586694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.590033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66d1b3-b195-4123-96d3-35a7f89fb56b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.613852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:43.626782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.627395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.627698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.627970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.628602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:43.628903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.630287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.635242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 505/999] 10.222.0.55 () {56 vars in 1122 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:43.641563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.701484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea34c96-d665-435d-bff7-f6c2ed252b3c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.715733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:43.739985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.740604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.740886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.741184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.741893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] RBAC: Adding query filter params (interface=public, service_id=07ebd6a115014ad99ed56907e2024e4c, region_id=RegionOne) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:43.742232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.743648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.749301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 495/1000] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/endpoints?service_id=07ebd6a115014ad99ed56907e2024e4c&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:43.755301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:43.767972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79268097-f607-459d-8500-b7db3fff15a2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.808776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf39d7-9060-40cf-b2aa-f9e43a05bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.809395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf39d7-9060-40cf-b2aa-f9e43a05bbe8 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.809673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf39d7-9060-40cf-b2aa-f9e43a05bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.809942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf39d7-9060-40cf-b2aa-f9e43a05bbe8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.810627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf39d7-9060-40cf-b2aa-f9e43a05bbe8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.812051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf39d7-9060-40cf-b2aa-f9e43a05bbe8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.839561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 506/1001] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:43.882216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe880-95be-4600-8625-d45a188200d8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:43.883267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fad19-0283-4bdf-8c84-4e82e469fcf1 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.883587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fad19-0283-4bdf-8c84-4e82e469fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.883878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fad19-0283-4bdf-8c84-4e82e469fcf1 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.885418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 507/1002] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:43.891358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.891720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.891952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.912761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:43.914004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:43.935404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:43.936058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:43.936332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:43.936597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:43.937210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:43.938613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:43.951465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 496/1003] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:43 2022] GET /identity/v3/services => generated 1727 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:32:43.969012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:44.001402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 508/1004] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:43 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:44.330717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:44.369294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:44.447955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb71a8-efac-4020-86f1-c384279f677b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:44.484568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:44.485207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:44.485539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:44.485816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] PATH_INFO: `/v3/services/image` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:44.486458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:44.487879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:44.498770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:44.499873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:44.500722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:44.503776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 497/1005] 10.222.0.55 () {56 vars in 1105 bytes} [Sat Sep 24 19:32:44 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:44.506838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:44.609918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ece5a-938f-4912-872a-189df8bb71a9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:44.649535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:44.650147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:44.650419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:44.650683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:44.651313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] RBAC: Adding query filter params (name=image) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:44.652157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:44.653613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e4f35-b3a4-4000-b14c-0e9bd6c3bf57 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:44.663457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 509/1006] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:44.664806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:44.776194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:44.780966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de122ea-4395-4338-84b1-b0fc9ca40699 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:44.816192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:44.816834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:44.817129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:44.817402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:44.818033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] RBAC: Adding query filter params (type=image) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:44.818338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:44.819758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0091840-062a-49bc-a136-aacb6463cd80 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:44.825458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 498/1007] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:46.057811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762dd9-6b60-46fd-8862-e5ecdcdbf3a5 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.057811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762dd9-6b60-46fd-8862-e5ecdcdbf3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.057811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762dd9-6b60-46fd-8862-e5ecdcdbf3a5 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.058507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 510/1008] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:46.064404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529b08-0aca-4d64-aef6-d85573b29a21 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.064945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529b08-0aca-4d64-aef6-d85573b29a21 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.064945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529b08-0aca-4d64-aef6-d85573b29a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.087463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8f529b08-0aca-4d64-aef6-d85573b29a21 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:46.088817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8f529b08-0aca-4d64-aef6-d85573b29a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:46.160232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 499/1009] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:46.196121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee81a4-d475-4012-8345-eb07ed3d7197 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.196449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee81a4-d475-4012-8345-eb07ed3d7197 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.196678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee81a4-d475-4012-8345-eb07ed3d7197 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.198155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 511/1010] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:46.204423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.204713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.204987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.224244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:46.225442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:46.310504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 500/1011] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:46.429267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee81a4-d475-4012-8345-eb07ed3d7197 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:46.476703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:46.538091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee81a4-d475-4012-8345-eb07ed3d7197 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:46.573027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:46.574135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.574414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.574690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.575864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:46.577293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:46.579046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 512/1012] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:46.585576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:46.586623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:46.614552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:46.699785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9c2d-0ce1-489a-bbd2-4ed8d6ff1de1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:46.724504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e34e5-2950-444f-b25c-5bf3f7597869 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:46.735448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:46.747053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:46.748011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.748288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.748554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.749209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:46.751140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:46.765464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:46.766100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.766374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.766641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.767221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:46.768617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:46.780219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:46.781773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 19:32:46.782868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:46.783720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 501/1013] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:46 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:46.788016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 513/1014] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:46.794614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:46.888547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:46.920545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1fa92-aff8-45e2-bdec-db35de996cf4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:46.961392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:46.962139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:46.962481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:46.962820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:46.963572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:46.963935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:46.965589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f589313-555a-452b-a934-8a0135fdef6c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:46.971494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 502/1015] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:46.977180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:47.031611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12695519-ff40-4d32-bf98-9cf40e76c61a None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:47.032070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12695519-ff40-4d32-bf98-9cf40e76c61a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:47.032349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12695519-ff40-4d32-bf98-9cf40e76c61a None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:47.034080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 503/1016] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:47.040720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:47.040995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:47.041262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:47.064937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:47.067486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:47.092252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994ad2a-74c4-4f8e-b794-703617667d58 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:47.133112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:47.133719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:47.133992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:47.134255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:47.134883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] RBAC: Adding query filter params (type=compute) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:47.135179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:47.136632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:47.142141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 514/1017] 10.222.0.55 () {56 vars in 1112 bytes} [Sat Sep 24 19:32:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:47.150377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 504/1018] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:47.150564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:47.200518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:47.265009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da485339-3e14-4128-a687-e57b2a62c9ff None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:47.299958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dfbfc-5c47-4cc5-9919-bc33bd25afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:47.300623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dfbfc-5c47-4cc5-9919-bc33bd25afff None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:47.300899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dfbfc-5c47-4cc5-9919-bc33bd25afff None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:47.301166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dfbfc-5c47-4cc5-9919-bc33bd25afff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:47.301925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dfbfc-5c47-4cc5-9919-bc33bd25afff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:47.303289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dfbfc-5c47-4cc5-9919-bc33bd25afff None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:47.334782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 515/1019] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:47.427622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:47.504241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:47.543375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689ed57-c881-42ac-9401-2718f1ba2b62 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:47.587879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6c89c-4274-4426-8856-b45caa9cc76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:47.588489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6c89c-4274-4426-8856-b45caa9cc76d None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:47.588763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6c89c-4274-4426-8856-b45caa9cc76d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:47.589155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6c89c-4274-4426-8856-b45caa9cc76d None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:47.590023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6c89c-4274-4426-8856-b45caa9cc76d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:47.591341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6c89c-4274-4426-8856-b45caa9cc76d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:47.608143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 505/1020] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:32:47 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:49.037725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fd1f-a6c3-4aad-89e1-90b10b4bd5b9 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.038328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fd1f-a6c3-4aad-89e1-90b10b4bd5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.038429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fd1f-a6c3-4aad-89e1-90b10b4bd5b9 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.039845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 516/1021] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:49.045704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.046146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.046241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.068317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:49.068317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:49.145258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 506/1022] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:49.418747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fd1f-a6c3-4aad-89e1-90b10b4bd5b9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:49.522842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:49.538005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fd1f-a6c3-4aad-89e1-90b10b4bd5b9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:49.576299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5eeaf-be98-4a74-871f-509e67a60d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:49.576941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5eeaf-be98-4a74-871f-509e67a60d0f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.577223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5eeaf-be98-4a74-871f-509e67a60d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.577497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5eeaf-be98-4a74-871f-509e67a60d0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.578657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5eeaf-be98-4a74-871f-509e67a60d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:49.580135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5eeaf-be98-4a74-871f-509e67a60d0f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:49.581752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 517/1023] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:49.587651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:49.644293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:49.650812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde9494-7cb1-413e-97e8-41fdf0ac317b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.651099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde9494-7cb1-413e-97e8-41fdf0ac317b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.651410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde9494-7cb1-413e-97e8-41fdf0ac317b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.652952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 518/1024] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:49.658157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b459f5-9afb-446f-8417-a2f7fa0314c0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.658443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b459f5-9afb-446f-8417-a2f7fa0314c0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.658707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b459f5-9afb-446f-8417-a2f7fa0314c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.677676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d7b459f5-9afb-446f-8417-a2f7fa0314c0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:49.678774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d7b459f5-9afb-446f-8417-a2f7fa0314c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:49.698362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26a696-a3b4-4cd1-ae9d-6fa55cc19a08 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:49.743799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651859ff-eaed-4103-a9c4-f0841d1fd5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:49.744547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651859ff-eaed-4103-a9c4-f0841d1fd5a0 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.744776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651859ff-eaed-4103-a9c4-f0841d1fd5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.746238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651859ff-eaed-4103-a9c4-f0841d1fd5a0 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.746946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651859ff-eaed-4103-a9c4-f0841d1fd5a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:49.748792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651859ff-eaed-4103-a9c4-f0841d1fd5a0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:49.761866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 519/1025] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:49.778140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 507/1026] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:32:49 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:49.869840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568c932-bc3e-4168-aa84-f342b6aa9e3d None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.870843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568c932-bc3e-4168-aa84-f342b6aa9e3d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.871211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568c932-bc3e-4168-aa84-f342b6aa9e3d None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.873805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 520/1027] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:49.879848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:49.880141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:49.880409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:49.900620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:49.902598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:49.986940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 508/1028] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:50.035283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568c932-bc3e-4168-aa84-f342b6aa9e3d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.135705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:50.148448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568c932-bc3e-4168-aa84-f342b6aa9e3d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.184784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.185417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.185690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.185955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.196956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:50.201997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:50.202969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:50.203857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 521/1029] 10.222.0.55 () {56 vars in 1120 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:50.210599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.254752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:50.261213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.329703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3bfc7-8eef-462e-8d31-83acf3e412bd None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.372721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.373468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.373730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.373999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.374667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:50.374970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.376098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4c59-7e9b-4cee-abf7-a1a7ef23a50a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.376383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.387462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 509/1030] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:50.391101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.421077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.421744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.422023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.422362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.422975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.424471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:50.434731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:50.435833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:50.437137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 522/1031] 10.222.0.55 () {56 vars in 1105 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:50.441856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.458591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:50.504001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:50.507332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28a83f-1b6a-4528-8877-e00401995c3a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.544127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.545050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.545326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.545738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.555183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4959fe-0db6-4093-b2fd-40468823a6d6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.561121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.562656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:50.578522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:50.579602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:50.580548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:50.580548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:50.580548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:50.580548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 510/1032] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:50.585355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.597138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.597550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.597824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.598091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.598717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] RBAC: Adding query filter params (name=image) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:50.599178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.600409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.606550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 523/1033] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:50.612127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.694374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6e966-45b8-4963-9ca9-0a36cb38dcb1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.708017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:50.733067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bdef6-655f-439e-b77a-a1dc5e69a453 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.733867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.734487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.734762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.735029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.735709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:50.736007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.737454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.751550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 511/1034] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:50.757406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.773589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:50.778735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.779369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.779672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.779941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.780587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:50.780884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.782291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.790785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 524/1035] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:50.797565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.869879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180730ca-7a35-4c21-8759-e5683e41ede0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.915606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.916236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.916509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.916771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.917961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b190a-e9cb-4497-9da2-8e4b196bc6b7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:50.925723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.927218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:50.940362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:50.941393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:50.942357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 512/1036] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:50.947391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.958018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:50.958632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:50.958934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:50.959207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:50.959940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] RBAC: Adding query filter params (interface=public, service_id=e936326dd0354e57b600828080557805, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:50.960251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:50.961617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:50.972500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 525/1037] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/endpoints?service_id=e936326dd0354e57b600828080557805&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:50.975353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:50.986646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:51.030721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:51.072378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7649e-1b3e-4595-b6af-bfd2ceeb74d2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:51.094839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca522ff-1606-4c56-8aa4-ceb1acb31986 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:51.123475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:51.123475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:51.123475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:51.123475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:51.123920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:51.123920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:51.123920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:51.133248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:51.133867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:51.134145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:51.134414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:51.135023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:51.136509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:51.146123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 513/1038] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:51.146312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 526/1039] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:32:50 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:32:51.146908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:51.270737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e04af-43f6-4ba2-abca-4b8e058ffe8e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:51.306367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:51.315452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:51.316107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:51.316387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:51.316658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:51.393298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:51.394767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:51.508547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 514/1040] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 19:32:51.510116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:51.587820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:51.623117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca454-4891-4678-818e-45044a36f525 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:51.661134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:51.661793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:51.662072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:51.662341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:51.674413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:51.675912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:51.692627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:51.693821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:51.695584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:51.695584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:51.695584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:51.695584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 527/1041] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:51.700411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:51.734809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:51.817650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93880e-ad0c-46ac-b5b3-94ff413cc438 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:51.853906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:51.854481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:51.855284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:51.855717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:51.856384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:51.856682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:51.858078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:51.875047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 515/1042] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:51.876425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:51.995584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80a28a-8523-4e4e-8152-18f6e5669845 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:52.020633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:52.033711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:52.034341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.034620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.034893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.041575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:52.043614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-58a1a7b1-90ee-4296-9d02-0f8b93967e45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:52.057563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:52.058913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:52.060254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:52.063087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 528/1043] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:52.064574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:52.092670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffe3c-a499-4a8f-9533-4cd96af141fd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.092974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffe3c-a499-4a8f-9533-4cd96af141fd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.093250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffe3c-a499-4a8f-9533-4cd96af141fd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.094592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 529/1044] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:52.100075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.100355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.100627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.123136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:52.124032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:52.169104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:52.179318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87088296-e349-4947-ba6f-427d93a073a8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:52.196573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 530/1045] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:52.220689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:52.221291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.221566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.221833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.222507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:52.222810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:52.224306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:52.236697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 516/1046] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:52.242525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:52.352925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323a5b0-7f09-4197-8e81-c044a376bae9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:52.366403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:52.389190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:52.389721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.390055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.390334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.391064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] RBAC: Adding query filter params (role.id=c00a7c03d65e424381c3a31a685c3016, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=57f68f991883493dbf78c0bcfd827c1d) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:52.391373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:52.392819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:52.398437 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 531/1047] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:52 2022] GET /identity/v3/role_assignments?user.id=57f68f991883493dbf78c0bcfd827c1d&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=c00a7c03d65e424381c3a31a685c3016 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:52.475444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:52.591790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cf337-7aec-4829-9d74-cd648d2a7fd0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:52.626689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:52.627321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.627629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.627904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.638182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:52.643110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:52.644163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:52.645034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 517/1048] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:52.650092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:52.755026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f53fb-4097-47a4-9800-1b16d2f8f5a1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:52.789323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:52.789923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.790201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.790465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.791121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:52.791414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:52.793353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:52.802356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 532/1049] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:52.808651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:52.823112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:52.922117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90395a5-2aad-4e66-a3c8-c581e09d6e3b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:52.958073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:52.958694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:52.958968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:52.959237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] PATH_INFO: `/v3/users/952b39192dab4ff68fb916c5b3c7dd55` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:52.973830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=952b39192dab4ff68fb916c5b3c7dd55)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:52.975280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:52.977156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 518/1050] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:32:52 2022] GET /identity/v3/users/952b39192dab4ff68fb916c5b3c7dd55 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:52.982671 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:53.022980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:53.095535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a4150-eb0d-4dc8-a69a-1e991bba721e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:53.130282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455de0c-d1dc-404f-8996-26632d1a14ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:53.130891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455de0c-d1dc-404f-8996-26632d1a14ab None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:53.131172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455de0c-d1dc-404f-8996-26632d1a14ab None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:53.131461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455de0c-d1dc-404f-8996-26632d1a14ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:53.202346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455de0c-d1dc-404f-8996-26632d1a14ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:53.203749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455de0c-d1dc-404f-8996-26632d1a14ab None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:53.249555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:53.326646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 533/1051] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:32:53.328010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:53.396142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:53.418556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8193da-dfcb-46fe-b1be-15fd953a0d27 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:53.418982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8193da-dfcb-46fe-b1be-15fd953a0d27 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:53.419349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8193da-dfcb-46fe-b1be-15fd953a0d27 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:53.421182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 534/1052] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:53.425876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:53.426161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:53.426431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:53.438021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae052a26-cb65-4b35-895c-1d31d02f3cb1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:53.446613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:53.447825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:53.474456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:53.475004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:53.475277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:53.475575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:53.476856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] RBAC: Authorizing `identity:get_project(project_id=7b631e0327b2445ab18387101c65feaf)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:53.478292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:53.480003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 519/1053] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:32:53 2022] GET /identity/v3/projects/7b631e0327b2445ab18387101c65feaf => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:53.486409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:53.525096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 535/1054] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:53.592765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da7bf-cdaf-4bac-a1a2-32840cbc108d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:53.628778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:53.629573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:53.629846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:53.630110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:53.630839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=7b631e0327b2445ab18387101c65feaf, user.id=952b39192dab4ff68fb916c5b3c7dd55) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:53.631143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:53.632731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:53.636927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:53.642176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 520/1055] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:32:53 2022] GET /identity/v3/role_assignments?user.id=952b39192dab4ff68fb916c5b3c7dd55&scope.project.id=7b631e0327b2445ab18387101c65feaf&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:53.817764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:53.883897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:53.919146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d519-bf97-467f-b1ee-3bd40a9743d8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:53.953719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:53.954438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:53.954781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:53.955109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] PATH_INFO: `/v3/services/image` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:53.955774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:53.957194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:53.966493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:53.967786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 19:32:53.968853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:53.970148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 536/1056] 10.222.0.55 () {56 vars in 1105 bytes} [Sat Sep 24 19:32:53 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:53.975759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:54.085957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa8ca4-b860-4938-bd8d-3e4636ebe2e8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:54.121423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:54.122032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:54.122305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:54.122569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:54.123191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] RBAC: Adding query filter params (name=image) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:54.123509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:54.124868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:54.130003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 521/1057] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:54.135400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:54.242360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a5486-f33f-4e84-b06a-52bb13b08f42 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:54.272260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:54.278538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:54.279143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:54.279418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:54.279708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:54.280333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] RBAC: Adding query filter params (type=image) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:54.280632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:54.281985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a53b8a-85e2-4f66-8bdf-68aaecdee98a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:54.291764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 537/1058] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:54.293363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:54.359371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:54.398630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311e71-0549-45ea-9f69-76a21e21651a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:54.435299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e5e86-4c67-4f3b-b273-13a36f0dcf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:54.435936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e5e86-4c67-4f3b-b273-13a36f0dcf7d None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:54.436219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e5e86-4c67-4f3b-b273-13a36f0dcf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:54.436490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e5e86-4c67-4f3b-b273-13a36f0dcf7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:54.437162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e5e86-4c67-4f3b-b273-13a36f0dcf7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:54.438571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e5e86-4c67-4f3b-b273-13a36f0dcf7d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:54.461753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 522/1059] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:32:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:32:54.571063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d8d1-db9d-43b4-aa8a-5834b6bbee35 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:54.571462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d8d1-db9d-43b4-aa8a-5834b6bbee35 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:54.571823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d8d1-db9d-43b4-aa8a-5834b6bbee35 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:54.573860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 538/1060] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:54.579487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:54.580016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:54.580016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:54.599956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:54.601168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:54.682381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 523/1061] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:54.952641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d8d1-db9d-43b4-aa8a-5834b6bbee35 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:55.061279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d8d1-db9d-43b4-aa8a-5834b6bbee35 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:55.097519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:55.098147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.098421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.098688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.109272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:55.114116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:55.115214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:55.116134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 539/1062] 10.222.0.55 () {56 vars in 1120 bytes} [Sat Sep 24 19:32:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:55.122173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:55.171227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:55.230523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688dd8e-7a68-4ccc-92d3-09f31788913c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:55.265804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:55.266415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.266691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.266965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.267657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:55.267960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:55.269379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:55.278317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 524/1063] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:32:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:55.284003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:55.342852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:55.391439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff39ca-0a3b-4956-8df6-e4a8d351f118 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:55.426432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:55.427051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.427328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.427624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.438514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:55.439988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:55.455343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:55.456443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:55.457522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:55.457522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:55.457522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:55.459512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 540/1064] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:55.462545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:55.564834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48aa14-894b-4c1e-a36f-9c1b21cb2e65 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:55.574041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:55.600947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:55.601607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.601808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.602077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.602709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:55.603009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:55.605000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:55.617157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 525/1065] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:55.622816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:55.730878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c9c0e-85ce-4640-9f39-bb373a9fe7f8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:55.740749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:55.766878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:55.767528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.767808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.768075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.775793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:55.777807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bdec30c1-ff55-40af-911b-e252bc10e6fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:32:55.789090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:55.790105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:32:55.791131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:55.795503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 541/1066] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:32:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:55.796779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:55.824957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a994e5-7666-49a2-b2e9-82bf34791a82 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.825148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a994e5-7666-49a2-b2e9-82bf34791a82 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.825412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a994e5-7666-49a2-b2e9-82bf34791a82 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.826739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 542/1067] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:32:55.831750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.832026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.832289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.850612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:55.851820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:55.906062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b19ac-66f3-4677-a1dd-ab8d3dca8068 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:55.931571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 543/1068] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:32:55.942793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:55.943544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:55.943919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:55.944285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:55.945102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:55.945505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:55.947057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:55.973943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 526/1069] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:32:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:55.975317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.077846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.081823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19580ee-0666-4e1d-a42e-f11d4452781f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.118687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.119495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.119932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.120378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.131247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.132830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:56.148362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:56.149511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:56.150613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:56.150613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:56.150613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:56.157072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 544/1070] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:56.158279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.189053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.192870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.270217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911c48a-ab62-42ec-9157-8b1df5c16997 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.310055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.310716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.311009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.311301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.312049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:56.312372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.313881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.322651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146b3ca-c30b-45e8-bb69-f7b2898936a1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.327536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 527/1071] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:56.333351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.359010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.359659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.359940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.360211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.371554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:56.376367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:56.377311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:56.380406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 545/1072] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:56.383652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.398242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.431471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.449194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11d065-43c1-4a4a-ad27-193de0087553 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.486123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.486828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.487137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.487462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.496059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.497513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.507357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f842-3bf2-4aab-8262-c6d659da3b36 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:32:56.512020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:56.518327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 528/1073] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:56.519656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.553552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.554903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.555262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.555558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.556334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:56.556670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.560016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.575968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 546/1074] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:56.578249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.635102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8671-8a1f-49fc-9c37-b91be02250dd None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.666419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.674989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.675953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.676231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.676501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.677193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:56.677499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.678936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30abaf1-a132-462c-8416-fcc411f478a3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.692371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 529/1075] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:56.696752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca129678-f4c0-47ca-a065-3adec501c1f9 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.697036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca129678-f4c0-47ca-a065-3adec501c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.697548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca129678-f4c0-47ca-a065-3adec501c1f9 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.698863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 530/1076] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:56.699570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca129678-f4c0-47ca-a065-3adec501c1f9 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.699963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cae3e4-ab68-4da8-9a84-5b4988a120ca None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.708188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.737043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b0227-a442-4ae8-873d-a02ab724d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.737662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b0227-a442-4ae8-873d-a02ab724d5cb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.737937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b0227-a442-4ae8-873d-a02ab724d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.738236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b0227-a442-4ae8-873d-a02ab724d5cb None admin] PATH_INFO: `/v3/users/952b39192dab4ff68fb916c5b3c7dd55` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.739352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b0227-a442-4ae8-873d-a02ab724d5cb None admin] RBAC: Authorizing `identity:get_user(user_id=952b39192dab4ff68fb916c5b3c7dd55)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.740792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b0227-a442-4ae8-873d-a02ab724d5cb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.742364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 547/1077] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/users/952b39192dab4ff68fb916c5b3c7dd55 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:32:56.743480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.743755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.744019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.762317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:56.763556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:56.805100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca129678-f4c0-47ca-a065-3adec501c1f9 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:56.831433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 548/1078] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:56.832302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:56.858613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb896285-e331-4c66-a605-73416161298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:56.859286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb896285-e331-4c66-a605-73416161298b None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:56.859622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb896285-e331-4c66-a605-73416161298b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:56.859872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb896285-e331-4c66-a605-73416161298b None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/57f68f991883493dbf78c0bcfd827c1d/roles/c00a7c03d65e424381c3a31a685c3016` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:56.862538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb896285-e331-4c66-a605-73416161298b None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=57f68f991883493dbf78c0bcfd827c1d, role_id=c00a7c03d65e424381c3a31a685c3016)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:56.864169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb896285-e331-4c66-a605-73416161298b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:56.879644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 531/1079] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:32:56 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/57f68f991883493dbf78c0bcfd827c1d/roles/c00a7c03d65e424381c3a31a685c3016 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:32:56.924133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:56.970786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5eab67-eb0b-4987-9393-cf8725de563b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:57.015242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:57.016069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:57.016346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:57.016621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:57.017931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] RBAC: Authorizing `identity:get_project(project_id=7b631e0327b2445ab18387101c65feaf)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:57.019508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:57.021440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 549/1080] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:32:56 2022] GET /identity/v3/projects/7b631e0327b2445ab18387101c65feaf => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:57.028486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb896285-e331-4c66-a605-73416161298b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:57.103120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:57.108378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:57.143759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb896285-e331-4c66-a605-73416161298b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:57.181521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:57.182112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:57.182407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:57.183179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf/users/952b39192dab4ff68fb916c5b3c7dd55/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:57.185157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b631e0327b2445ab18387101c65feaf, user_id=952b39192dab4ff68fb916c5b3c7dd55, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:57.186584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:57.201770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 532/1081] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:32:57 2022] PUT /identity/v3/projects/7b631e0327b2445ab18387101c65feaf/users/952b39192dab4ff68fb916c5b3c7dd55/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:32:57.218583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a74c9-7978-46e5-a344-5eaac066eee4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:57.271092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:57.271828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:57.272142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:57.272458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:57.283976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:57.356747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:57.358231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:57.471862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 550/1082] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:32:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 19:32:57.477351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:57.580254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b2e2c-77e3-4120-ad09-5408676f5f86 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:57.607237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:57.617967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:57.618599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:57.618873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:57.619141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:57.629916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:57.631370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-09a45d81-b4a0-4c16-8721-187712c6ee7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:57.646161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:57.648023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:57.648924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:57.648924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:57.648924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:57.651562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 533/1083] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:57.653850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:57.695519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:57.757181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a340eb-a287-4d0e-ac51-bef7279ebb52 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:57.790340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:57.790957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:57.791232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:57.791521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:57.792692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:57.793170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:57.794566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb32e26-086d-4277-9d9e-df43de9305d4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:57.815451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 551/1084] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:59.079313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64569-e27f-4ccf-bb24-5a20529564a8 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.079835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64569-e27f-4ccf-bb24-5a20529564a8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.079926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64569-e27f-4ccf-bb24-5a20529564a8 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.081374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 534/1085] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:59.086502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460656-6e7a-4e66-ba80-0a35183286e3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.086931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460656-6e7a-4e66-ba80-0a35183286e3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.087133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460656-6e7a-4e66-ba80-0a35183286e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.105832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5c460656-6e7a-4e66-ba80-0a35183286e3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:59.107018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5c460656-6e7a-4e66-ba80-0a35183286e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:59.172852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 552/1086] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:59.402640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b815e-45d9-4b72-9520-9b8876c75c1d None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.402920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b815e-45d9-4b72-9520-9b8876c75c1d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.403189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b815e-45d9-4b72-9520-9b8876c75c1d None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.404562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 535/1087] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:32:59.409993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.410301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.410573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.428780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:32:59.430206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:32:59.442243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b815e-45d9-4b72-9520-9b8876c75c1d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:59.453357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:59.503837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 553/1088] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:32:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:32:59.547302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b815e-45d9-4b72-9520-9b8876c75c1d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:59.580849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:59.581471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.581748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.582018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.592393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:59.597131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:59.598233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:59.599261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 536/1089] 10.222.0.55 () {56 vars in 1120 bytes} [Sat Sep 24 19:32:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:59.604360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:59.701995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39137e23-f66f-4020-a61d-bcd369af83c1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:59.735689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:59.736294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.736571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.736839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.737496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:32:59.737797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:59.739154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:59.747830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 554/1090] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:32:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:32:59.753858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:59.776789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:59.795723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:59.798986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:32:59.856433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294daf-dfb0-4bc5-bf87-5576bf864beb None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:59.878900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87044ad5-0209-42f7-a901-8ea9f751e641 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:32:59.891238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:59.891913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.892236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.892511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.904160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:32:59.905777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:32:59.912650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:32:59.913274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:32:59.913552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:32:59.913815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:32:59.920746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:59.921877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:32:59.922973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:32:59.922973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:32:59.922973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:32:59.922973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 537/1091] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:32:59.927659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:32:59.928302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:32:59.934592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:32:59.935806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:32:59.936974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 555/1092] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:32:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:32:59.941420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.021177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.023243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.035355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966599c-b260-4dd5-86e9-ec5862f6b286 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.042393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3a4e-d5c6-4944-b70a-4a458d97ca73 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.078301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.078919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.079197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.079508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.080159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:00.080444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.081496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.081818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.082193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.082500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.082800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.083601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:00.083972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.085409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.094126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 538/1093] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:00.094273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 556/1094] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:32:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:00.099880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.100003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.201622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ad8b-601e-4211-b41a-562e670fa44f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.205336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1f28-1de9-467c-8519-a6100d75c2a6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.235141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.235822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.236121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.236393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] PATH_INFO: `/v3/users/952b39192dab4ff68fb916c5b3c7dd55` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.237522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] RBAC: Authorizing `identity:get_user(user_id=952b39192dab4ff68fb916c5b3c7dd55)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.238345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.238951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.239045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.239232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.239527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.240592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 557/1095] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/users/952b39192dab4ff68fb916c5b3c7dd55 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:00.246905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.247519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.249053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.258993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:33:00.261619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:00.263035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:33:00.264186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 539/1096] 10.222.0.55 () {56 vars in 1093 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:00.268436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.281278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.352520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156686ad-d9c9-4f51-8d18-ec034f3f6a2e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.367867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f58b8-ef5d-47be-a795-a7b44c5ad12c None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.387969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.388575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.388856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.389128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.404049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.404793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.405090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.405367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.406217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:00.406568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.408283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.427236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 540/1097] 10.222.0.55 () {56 vars in 1133 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:00.433748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.464696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.466215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.477208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.501453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.535937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be798e-d893-471b-90ee-30b5dbd148a8 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.567795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 558/1098] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:00.573826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.573964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.574565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.574843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.575114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.643028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.644516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.671341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058cebd-8b7a-4392-8558-a65c9d596c4c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.683723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.708581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.709284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.709559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.709822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.710916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] RBAC: Authorizing `identity:get_project(project_id=7b631e0327b2445ab18387101c65feaf)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.712354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.712874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:00.714719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 559/1099] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/projects/7b631e0327b2445ab18387101c65feaf => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:00.721454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.752281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 541/1100] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:00.757980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.822556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8655c-d318-462c-b8a8-152e7024b721 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.855295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade247-05e2-493a-9f7e-f6be1fe0a6ce None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:00.856692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.857287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.857562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.857824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.858540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=7b631e0327b2445ab18387101c65feaf, user.id=952b39192dab4ff68fb916c5b3c7dd55) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:00.858842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.860283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.866114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 560/1101] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/role_assignments?user.id=952b39192dab4ff68fb916c5b3c7dd55&scope.project.id=7b631e0327b2445ab18387101c65feaf&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:00.891185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:00.891863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:00.892134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:00.892396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:00.904243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:00.905722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:33:00.921640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:00.922750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:33:00.923795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:33:00.923795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:33:00.923795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:33:00.923795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 542/1102] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:33:00.928879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:00.976219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:01.036333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29adb7-90bb-486f-bca1-74851fda9b08 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:01.072359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:01.074291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:01.074291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:01.074291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:01.074807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:01.075121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:01.076566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:01.088863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 561/1103] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:01.094702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:01.116789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:01.194804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545126b-6492-4b65-94f8-e395fefde224 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:01.227866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:01.228495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:01.228773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:01.229043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:01.236289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:01.237732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:33:01.249581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:01.250472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:33:01.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 543/1104] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:01.256466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:01.358617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37154bc0-abce-4789-b691-dcbbf2f95094 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:01.360106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:01.394812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:01.395415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:01.395710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:01.395978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:01.396657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:01.396961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:01.398379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c8ee4-6a54-469e-a486-c20d130361a8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:01.410304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 562/1105] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:01.416334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:01.498433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:01.517545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770769d-df70-4c7e-afb8-5d7002d8501d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:01.550688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:01.551286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:01.551599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:01.551870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:01.552584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] RBAC: Adding query filter params (role.id=c00a7c03d65e424381c3a31a685c3016, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=57f68f991883493dbf78c0bcfd827c1d) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:01.552887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:01.554314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301644b3-a617-4834-a1d2-e9199fe88e4c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:01.561104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 544/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:01 2022] GET /identity/v3/role_assignments?user.id=57f68f991883493dbf78c0bcfd827c1d&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=c00a7c03d65e424381c3a31a685c3016 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:03.077537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbe19b-08f2-44d7-844d-6bac19df65b6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:03.078088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbe19b-08f2-44d7-844d-6bac19df65b6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:03.078088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbe19b-08f2-44d7-844d-6bac19df65b6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:03.079489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 563/1107] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:03.087400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:03.088066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:03.088066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:03.107253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:03.108454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:03.172466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 545/1108] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:03.448337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbe19b-08f2-44d7-844d-6bac19df65b6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:03.549364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbe19b-08f2-44d7-844d-6bac19df65b6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:03.582879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa6a47-1cf1-44e0-9a22-56ec4c73ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:03.583520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa6a47-1cf1-44e0-9a22-56ec4c73ae7c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:03.583803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa6a47-1cf1-44e0-9a22-56ec4c73ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:03.584075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa6a47-1cf1-44e0-9a22-56ec4c73ae7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:03.585196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa6a47-1cf1-44e0-9a22-56ec4c73ae7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:03.586625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa6a47-1cf1-44e0-9a22-56ec4c73ae7c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:03.588208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 564/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:33:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:03.594080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:03.668949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:03.693753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6561b-8d04-4401-abc2-bbef23d0b8bb None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:03.727835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe7750-bdc6-4f79-8fd0-d5d8474bfce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:03.728366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe7750-bdc6-4f79-8fd0-d5d8474bfce1 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:03.728643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe7750-bdc6-4f79-8fd0-d5d8474bfce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:03.728911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe7750-bdc6-4f79-8fd0-d5d8474bfce1 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:03.730113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe7750-bdc6-4f79-8fd0-d5d8474bfce1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:03.731727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe7750-bdc6-4f79-8fd0-d5d8474bfce1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:03.752555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 546/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:33:03 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:33:06.080389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757a8d2-01d6-4353-a08e-070426745580 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:06.080943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757a8d2-01d6-4353-a08e-070426745580 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:06.080943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757a8d2-01d6-4353-a08e-070426745580 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:06.082316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 565/1111] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:06.087500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:06.087933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:06.088041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:06.106227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:06.107411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:06.173606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 547/1112] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:06.446947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757a8d2-01d6-4353-a08e-070426745580 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:06.477188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:06.549415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757a8d2-01d6-4353-a08e-070426745580 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:06.587180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:06.587826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:06.588099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:06.588391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] PATH_INFO: `/v3/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:06.589797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] RBAC: Authorizing `identity:get_role(role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:06.591144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:06.592498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 566/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:06 2022] GET /identity/v3/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:06.598541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:06.657985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:06.703526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973bc39-25c6-4060-8abf-458451f98aca None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:06.740266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:06.741162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:06.741619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:06.742106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] PATH_INFO: `/v3/users/a00cde2c927d4e62b1a0e1a8d488c620` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:06.757106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] RBAC: Authorizing `identity:get_user(user_id=a00cde2c927d4e62b1a0e1a8d488c620)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:06.758797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:06.761421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 548/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:06 2022] GET /identity/v3/users/a00cde2c927d4e62b1a0e1a8d488c620 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:06.767410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:06.877371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:06.881620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb28a-d8de-413a-8779-8b9a29ad749d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:06.917472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:06.918065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:06.918339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:06.918603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:06.985185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:06.986560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:07.080895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:07.086169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 567/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:07.092747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:07.195503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6276-17ce-46e8-94a1-8aeb6895fea0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:07.214005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:07.231927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015e599-da43-4495-8e4d-318b7062004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:07.232558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e599-da43-4495-8e4d-318b7062004d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:07.232829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e599-da43-4495-8e4d-318b7062004d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:07.233095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e599-da43-4495-8e4d-318b7062004d None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:07.234197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015e599-da43-4495-8e4d-318b7062004d None admin] RBAC: Authorizing `identity:get_project(project_id=7b631e0327b2445ab18387101c65feaf)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:07.235711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015e599-da43-4495-8e4d-318b7062004d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:07.237289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 549/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:07 2022] GET /identity/v3/projects/7b631e0327b2445ab18387101c65feaf => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:07.244172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:07.358130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e93d1-0952-4830-bbc2-6af7af2cc110 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:07.393617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:07.394329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:07.394622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:07.394945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:07.395716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] RBAC: Adding query filter params (role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=7b631e0327b2445ab18387101c65feaf, user.id=a00cde2c927d4e62b1a0e1a8d488c620) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:07.396080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:07.397780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709fd08-a001-4d4d-9bf5-5d6854e63e82 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:07.404019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 568/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:07 2022] GET /identity/v3/role_assignments?user.id=a00cde2c927d4e62b1a0e1a8d488c620&scope.project.id=7b631e0327b2445ab18387101c65feaf&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:09.578080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1d08-56c5-4c43-8ce5-73242bfcdd79 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:09.578663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1d08-56c5-4c43-8ce5-73242bfcdd79 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:09.578663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1d08-56c5-4c43-8ce5-73242bfcdd79 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:09.579994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 550/1118] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:09.585330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:09.585743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:09.585884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:09.604618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:09.606061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:09.666266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:09.683180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 569/1119] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:09.958079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab1d08-56c5-4c43-8ce5-73242bfcdd79 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:10.063132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab1d08-56c5-4c43-8ce5-73242bfcdd79 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:10.099874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:10.100684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:10.101003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:10.101301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] PATH_INFO: `/v3/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:10.105111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] RBAC: Authorizing `identity:get_role(role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:10.105111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:10.106351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 551/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:09 2022] GET /identity/v3/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:10.112641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:10.218064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42c27a-9036-4823-9387-4ccab4b6a227 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:10.255192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:10.255850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:10.256125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:10.256411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] PATH_INFO: `/v3/users/a00cde2c927d4e62b1a0e1a8d488c620` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:10.257511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] RBAC: Authorizing `identity:get_user(user_id=a00cde2c927d4e62b1a0e1a8d488c620)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:10.259065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:10.261214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 570/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:10 2022] GET /identity/v3/users/a00cde2c927d4e62b1a0e1a8d488c620 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:10.267328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:10.325465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:10.376409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be6578-78c8-4d0a-9182-4a1cf24441aa None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:10.413139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7f4e3-f3de-4805-a4d5-f520af7039ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:10.413770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7f4e3-f3de-4805-a4d5-f520af7039ef None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:10.414048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7f4e3-f3de-4805-a4d5-f520af7039ef None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:10.414320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7f4e3-f3de-4805-a4d5-f520af7039ef None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:10.415486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7f4e3-f3de-4805-a4d5-f520af7039ef None admin] RBAC: Authorizing `identity:get_project(project_id=7b631e0327b2445ab18387101c65feaf)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:10.417275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7f4e3-f3de-4805-a4d5-f520af7039ef None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:10.418933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 552/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:10 2022] GET /identity/v3/projects/7b631e0327b2445ab18387101c65feaf => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:33:10.425883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:10.497762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:10.541510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036a6f2-d3d6-4258-901e-6e54f7f2bbba None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:10.583277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4551d94-fa7e-4da6-9d19-309ba01d157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:10.584125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4551d94-fa7e-4da6-9d19-309ba01d157f None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:10.584480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4551d94-fa7e-4da6-9d19-309ba01d157f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:10.584842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4551d94-fa7e-4da6-9d19-309ba01d157f None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf/users/a00cde2c927d4e62b1a0e1a8d488c620/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:10.587516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4551d94-fa7e-4da6-9d19-309ba01d157f None admin] RBAC: Authorizing `identity:create_grant(project_id=7b631e0327b2445ab18387101c65feaf, user_id=a00cde2c927d4e62b1a0e1a8d488c620, role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:10.589380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4551d94-fa7e-4da6-9d19-309ba01d157f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:10.603009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 571/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:33:10 2022] PUT /identity/v3/projects/7b631e0327b2445ab18387101c65feaf/users/a00cde2c927d4e62b1a0e1a8d488c620/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:33:12.830176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cf1ca-5e7f-4ba3-8f14-b4c7ffe3301e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:12.830741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cf1ca-5e7f-4ba3-8f14-b4c7ffe3301e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:12.830741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cf1ca-5e7f-4ba3-8f14-b4c7ffe3301e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:12.832117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 553/1124] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:12.837282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:12.837685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:12.837830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:12.856055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:12.857249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:12.924496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 572/1125] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:13.202679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cf1ca-5e7f-4ba3-8f14-b4c7ffe3301e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:13.307045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cf1ca-5e7f-4ba3-8f14-b4c7ffe3301e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:13.329368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:13.341072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:13.341693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:13.341966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:13.342232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] PATH_INFO: `/v3/roles/492c8fef29584d08b9edfb06d2d7d1c6` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:13.343630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] RBAC: Authorizing `identity:get_role(role_id=492c8fef29584d08b9edfb06d2d7d1c6)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:13.345116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:13.346439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 554/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:13 2022] GET /identity/v3/roles/492c8fef29584d08b9edfb06d2d7d1c6 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:13.352862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:13.403820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:13.454670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486a485-8450-4171-9f07-47afd4ef1613 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:13.487072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:13.487725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:13.488028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:13.488270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] PATH_INFO: `/v3/users/a00cde2c927d4e62b1a0e1a8d488c620` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:13.489380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] RBAC: Authorizing `identity:get_user(user_id=a00cde2c927d4e62b1a0e1a8d488c620)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:13.490806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:13.492410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 573/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:13 2022] GET /identity/v3/users/a00cde2c927d4e62b1a0e1a8d488c620 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:13.498760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:13.595715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13580705-2e57-4c78-a288-7d7bf02de13e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:13.628790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:13.629390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:13.629665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:13.629933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:13.685464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:13.695898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:13.697353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:13.798104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 555/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:13.803980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:13.902594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47537f9f-fb7e-439d-92c1-2dbf7f181091 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:13.921802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:13.939903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2db9d-7245-4416-a67e-b3cd6f4451ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:13.940533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2db9d-7245-4416-a67e-b3cd6f4451ff None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:13.940806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2db9d-7245-4416-a67e-b3cd6f4451ff None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:13.941072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2db9d-7245-4416-a67e-b3cd6f4451ff None admin] PATH_INFO: `/v3/projects/7b631e0327b2445ab18387101c65feaf` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:13.942170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2db9d-7245-4416-a67e-b3cd6f4451ff None admin] RBAC: Authorizing `identity:get_project(project_id=7b631e0327b2445ab18387101c65feaf)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:13.943617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2db9d-7245-4416-a67e-b3cd6f4451ff None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:13.945246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 574/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:13 2022] GET /identity/v3/projects/7b631e0327b2445ab18387101c65feaf => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:13.952371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:14.045720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:14.055730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23087f42-4455-4406-bd16-c16aa62af0e1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:14.092301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:14.092913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:14.093194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:14.093467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:14.094188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] RBAC: Adding query filter params (role.id=492c8fef29584d08b9edfb06d2d7d1c6, scope.project.id=7b631e0327b2445ab18387101c65feaf, user.id=a00cde2c927d4e62b1a0e1a8d488c620) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:14.094499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:14.095990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491f2f6-bac3-4710-8f07-b5bdb8bdc541 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:14.101884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 556/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:13 2022] GET /identity/v3/role_assignments?user.id=a00cde2c927d4e62b1a0e1a8d488c620&scope.project.id=7b631e0327b2445ab18387101c65feaf&role.id=492c8fef29584d08b9edfb06d2d7d1c6 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:16.382786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7184e-86b3-4904-aef5-07bed82feb29 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:16.383370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7184e-86b3-4904-aef5-07bed82feb29 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:16.383370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7184e-86b3-4904-aef5-07bed82feb29 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:16.384613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 575/1131] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:16.390360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:16.390812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:16.390918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:16.409747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:16.410935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:16.480150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 557/1132] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:16.747543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7184e-86b3-4904-aef5-07bed82feb29 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:16.846086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7184e-86b3-4904-aef5-07bed82feb29 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:16.879395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bc3f7-62d7-401a-b3d3-6288839fcbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:16.880036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bc3f7-62d7-401a-b3d3-6288839fcbfa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:16.880310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bc3f7-62d7-401a-b3d3-6288839fcbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:16.880574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bc3f7-62d7-401a-b3d3-6288839fcbfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:16.881667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bc3f7-62d7-401a-b3d3-6288839fcbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:16.883079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bc3f7-62d7-401a-b3d3-6288839fcbfa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:16.884853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 576/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:33:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:16.890723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:16.974692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:16.990464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b954c6-c3a1-4381-aec0-304685e71d03 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:17.024130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbea6b-bb75-42bc-afb8-7b4e2779943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:17.024661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbea6b-bb75-42bc-afb8-7b4e2779943e None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:17.024938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbea6b-bb75-42bc-afb8-7b4e2779943e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:17.025205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbea6b-bb75-42bc-afb8-7b4e2779943e None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:17.026521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbea6b-bb75-42bc-afb8-7b4e2779943e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:17.027999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbea6b-bb75-42bc-afb8-7b4e2779943e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:17.055216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 558/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:33:16 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:33:19.303936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656758d4-76fe-4ac5-a7ca-4ef55120d269 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:19.304743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656758d4-76fe-4ac5-a7ca-4ef55120d269 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:19.304743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656758d4-76fe-4ac5-a7ca-4ef55120d269 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:19.306058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 577/1135] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:19.311465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:19.311910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:19.312005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:19.330523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:19.331751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:19.399122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 559/1136] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:19.667667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656758d4-76fe-4ac5-a7ca-4ef55120d269 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:19.726068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:19.778083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656758d4-76fe-4ac5-a7ca-4ef55120d269 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:19.811683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f53155-2396-4457-9d67-c657289cd6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:19.812306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f53155-2396-4457-9d67-c657289cd6ca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:19.812587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f53155-2396-4457-9d67-c657289cd6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:19.812867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f53155-2396-4457-9d67-c657289cd6ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:19.813990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f53155-2396-4457-9d67-c657289cd6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:19.815462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f53155-2396-4457-9d67-c657289cd6ca None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:19.817038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 578/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:33:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:19.824428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:19.896689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:19.933569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53186c-5254-4da4-a7cd-e888f6435103 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:19.966266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac5b4f-424d-450d-9256-c19038ec277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:19.966867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac5b4f-424d-450d-9256-c19038ec277c None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:19.967140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac5b4f-424d-450d-9256-c19038ec277c None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:19.967470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac5b4f-424d-450d-9256-c19038ec277c None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:19.968118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac5b4f-424d-450d-9256-c19038ec277c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:19.969525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac5b4f-424d-450d-9256-c19038ec277c None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:19.991843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 560/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:33:19 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:33:22.269114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce82503-06a8-4c68-a631-6b1abf3245a2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:22.269670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce82503-06a8-4c68-a631-6b1abf3245a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:22.269670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce82503-06a8-4c68-a631-6b1abf3245a2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:22.270983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 579/1139] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:22.276791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:22.277259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:22.277359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:22.295490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:22.296694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:22.365184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 561/1140] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:22.637203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce82503-06a8-4c68-a631-6b1abf3245a2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:22.741570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce82503-06a8-4c68-a631-6b1abf3245a2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:22.765389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:22.777454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:22.778079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:22.778354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:22.778619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:22.793366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:33:22.801187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:22.803903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:33:22.805324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 580/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:33:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:22.809890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:22.863392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:22.910635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427832b-320b-4d4e-83aa-9ab9dca13648 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:22.946820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:22.947496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:22.947775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:22.948039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:22.948692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:22.948988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:22.950356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:22.958277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 562/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:33:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:22.964913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:23.072577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d46d98-2da6-4043-b37e-85ddd95a7cd6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:23.106740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:23.107355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:23.107658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:23.107924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] PATH_INFO: `/v3/users/06246007a36a423491bd912d382d87b9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:23.121448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] RBAC: Authorizing `identity:get_user(user_id=06246007a36a423491bd912d382d87b9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:23.122881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:23.124518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 581/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:22 2022] GET /identity/v3/users/06246007a36a423491bd912d382d87b9 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:23.130239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:23.229415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0107e-c643-40d8-a85a-5a099ba61e67 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:23.242378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:23.262782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:23.263381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:23.263688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:23.263956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:23.329923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:23.331372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:23.428317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 563/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:23.434791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:23.476141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:23.542130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d08b3b-76f1-4d64-b174-3e2972b5857b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:23.576271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0e57-e830-45f8-888c-c7b4ef87c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:23.576895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0e57-e830-45f8-888c-c7b4ef87c651 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:23.577172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0e57-e830-45f8-888c-c7b4ef87c651 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:23.577443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0e57-e830-45f8-888c-c7b4ef87c651 None admin] PATH_INFO: `/v3/projects/101ab43310b4418b9d7472391a12881c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:23.578552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0e57-e830-45f8-888c-c7b4ef87c651 None admin] RBAC: Authorizing `identity:get_project(project_id=101ab43310b4418b9d7472391a12881c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:23.580016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0e57-e830-45f8-888c-c7b4ef87c651 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:23.581647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 582/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:23 2022] GET /identity/v3/projects/101ab43310b4418b9d7472391a12881c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:23.588750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:23.603480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:23.687900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8b8d-f9a5-483f-b765-a94ccd2d68d2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:23.721805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:23.722406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:23.722685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:23.722956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:23.723726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=101ab43310b4418b9d7472391a12881c, user.id=06246007a36a423491bd912d382d87b9) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:23.724031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:23.725497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc970c47-875c-41fc-8e1d-a5f404031e09 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:23.731483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 564/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:23 2022] GET /identity/v3/role_assignments?user.id=06246007a36a423491bd912d382d87b9&scope.project.id=101ab43310b4418b9d7472391a12881c&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:25.936745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9685-5990-4783-ac71-3c2d06dfc122 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:25.937297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9685-5990-4783-ac71-3c2d06dfc122 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:25.937297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9685-5990-4783-ac71-3c2d06dfc122 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:25.938626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 583/1147] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:25.943778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:25.944210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:25.944314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:25.962604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:25.964478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:26.030047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 565/1148] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:26.296914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9685-5990-4783-ac71-3c2d06dfc122 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:26.396340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9685-5990-4783-ac71-3c2d06dfc122 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:26.406117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:26.435410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:26.436073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:26.436349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:26.436620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:26.446685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:33:26.452019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:26.453670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:33:26.455308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 584/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:33:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:26.459276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:26.474943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:26.568396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e127f66-c90b-4ef1-b1e9-fa763a66099d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:26.601819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:26.602426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:26.602706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:26.602979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:26.604395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:26.604865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:26.606249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:26.614118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 566/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:33:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:26.620256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:26.719263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a066-ebae-4154-8b6b-bfec10fb85a5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:26.753395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:26.754018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:26.754294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:26.754562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] PATH_INFO: `/v3/users/06246007a36a423491bd912d382d87b9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:26.755721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] RBAC: Authorizing `identity:get_user(user_id=06246007a36a423491bd912d382d87b9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:26.757147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:26.758744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 585/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:26 2022] GET /identity/v3/users/06246007a36a423491bd912d382d87b9 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:26.765248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:26.839476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:26.870979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688850f-14f0-4e50-9217-bc6d1634978f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:26.905931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301145-7b4a-4f87-97e7-1ec3acc590ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:26.906572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301145-7b4a-4f87-97e7-1ec3acc590ff None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:26.906854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301145-7b4a-4f87-97e7-1ec3acc590ff None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:26.907127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301145-7b4a-4f87-97e7-1ec3acc590ff None admin] PATH_INFO: `/v3/projects/101ab43310b4418b9d7472391a12881c` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:26.908321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96301145-7b4a-4f87-97e7-1ec3acc590ff None admin] RBAC: Authorizing `identity:get_project(project_id=101ab43310b4418b9d7472391a12881c)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:26.909783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96301145-7b4a-4f87-97e7-1ec3acc590ff None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:26.911490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 567/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:26 2022] GET /identity/v3/projects/101ab43310b4418b9d7472391a12881c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:26.918228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:26.954002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:27.027686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e4c49-222c-49d2-b6ab-9cff260b58af None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:27.062251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0689e-1e95-46cc-b9e9-4e3a30525064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:27.062912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0689e-1e95-46cc-b9e9-4e3a30525064 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:27.063189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0689e-1e95-46cc-b9e9-4e3a30525064 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:27.063489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0689e-1e95-46cc-b9e9-4e3a30525064 None admin] PATH_INFO: `/v3/projects/101ab43310b4418b9d7472391a12881c/users/06246007a36a423491bd912d382d87b9/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:27.065391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0689e-1e95-46cc-b9e9-4e3a30525064 None admin] RBAC: Authorizing `identity:create_grant(project_id=101ab43310b4418b9d7472391a12881c, user_id=06246007a36a423491bd912d382d87b9, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:27.067675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0689e-1e95-46cc-b9e9-4e3a30525064 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:27.084100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 586/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:33:26 2022] PUT /identity/v3/projects/101ab43310b4418b9d7472391a12881c/users/06246007a36a423491bd912d382d87b9/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:33:29.282459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beca85-0845-46b6-9fff-9483cfd7c453 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:29.283017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beca85-0845-46b6-9fff-9483cfd7c453 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:29.283017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beca85-0845-46b6-9fff-9483cfd7c453 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:29.284372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 568/1154] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:29.289859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:29.290281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:29.290379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:29.309972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:29.311078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:29.376714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 587/1155] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:29.649544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3beca85-0845-46b6-9fff-9483cfd7c453 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:29.752013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3beca85-0845-46b6-9fff-9483cfd7c453 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:29.785368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:29.785990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:29.786272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:29.786544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:29.793127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:29.797563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:33:29.802539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:29.803667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:33:29.805008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 569/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:33:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:29.810185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:29.836518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:29.908190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccb277-0537-442a-89a2-6b3397c1b547 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:29.940413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:29.941005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:29.941279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:29.941544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:29.942191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:29.942489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:29.943867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:29.952365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 588/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:33:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:29.958454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:30.056389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5993d-3734-4554-9efe-665df800d023 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:30.090587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:30.091229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:30.091533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:30.091806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] PATH_INFO: `/v3/users/06246007a36a423491bd912d382d87b9` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:30.092916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] RBAC: Authorizing `identity:get_user(user_id=06246007a36a423491bd912d382d87b9)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:30.094352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:30.095969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 570/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:29 2022] GET /identity/v3/users/06246007a36a423491bd912d382d87b9 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:30.101693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:30.188395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:30.201541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6612-769e-44d6-9674-8fb69714ba3b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:30.233358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:30.233954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:30.234230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:30.234497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:30.299203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:30.300669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:30.387860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:30.400807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 589/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:30.406887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:30.475109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:30.511372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87068a3-761b-4f23-9d59-459b5fae2cca None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:30.544197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a361482-3a3e-4d78-b2dd-1ce8ff78a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:30.544818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a361482-3a3e-4d78-b2dd-1ce8ff78a093 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:30.545100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a361482-3a3e-4d78-b2dd-1ce8ff78a093 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:30.545368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a361482-3a3e-4d78-b2dd-1ce8ff78a093 None admin] PATH_INFO: `/v3/projects/101ab43310b4418b9d7472391a12881c` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:30.546461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a361482-3a3e-4d78-b2dd-1ce8ff78a093 None admin] RBAC: Authorizing `identity:get_project(project_id=101ab43310b4418b9d7472391a12881c)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:30.548003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a361482-3a3e-4d78-b2dd-1ce8ff78a093 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:30.549628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 571/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:30 2022] GET /identity/v3/projects/101ab43310b4418b9d7472391a12881c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:30.556066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:30.655261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b042b-c762-46fb-a807-5f1ea649375b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:30.690133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:30.690938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:30.691220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:30.691534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:30.692304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=101ab43310b4418b9d7472391a12881c, user.id=06246007a36a423491bd912d382d87b9) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:30.692680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:30.694388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeba27-5dd3-402b-a29f-334f7bb7e205 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:30.700777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 590/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:30 2022] GET /identity/v3/role_assignments?user.id=06246007a36a423491bd912d382d87b9&scope.project.id=101ab43310b4418b9d7472391a12881c&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:32.893491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f96ec3-5802-4abd-b49d-a3baf4ba5cc4 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:32.894061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f96ec3-5802-4abd-b49d-a3baf4ba5cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:32.894061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f96ec3-5802-4abd-b49d-a3baf4ba5cc4 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:32.895358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 572/1162] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:32.901340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:32.901887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:32.901887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:32.919996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:32.921203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:32.970020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:32.990291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 591/1163] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:33.261204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f96ec3-5802-4abd-b49d-a3baf4ba5cc4 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:33.363894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f96ec3-5802-4abd-b49d-a3baf4ba5cc4 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:33.398011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:33.398613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:33.398887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:33.399155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:33.401957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:33.470864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:33.472337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:33.572123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 573/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 19:33:33.578392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:33.690385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aff245-f0d5-4f9a-8e5b-2a5bb322b808 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:33.725525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:33.726157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:33.726432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:33.726700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:33.733802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:33.735246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9f1d7485-9003-4cd2-9801-99e7cfdbbea8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:33.746651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:33.747970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:33:33.748937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 592/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:33:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:33.754067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:33.788602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:33.856520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76990-a471-4405-86ce-225ff1894c93 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:33.892452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:33.893052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:33.893327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:33.893596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:33.894239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:33.894540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:33.895939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128de19-d4fc-4f5a-9881-3e4f447916ab None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:33.904054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 574/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Sat Sep 24 19:33:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:36.052390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691be1a-7e28-421e-9455-2b702ecf8a78 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:36.052930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691be1a-7e28-421e-9455-2b702ecf8a78 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:36.052930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691be1a-7e28-421e-9455-2b702ecf8a78 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:36.054249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 593/1167] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:36.060659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7e029-e80f-4ada-a790-b1beb38ebb29 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:36.061042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7e029-e80f-4ada-a790-b1beb38ebb29 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:36.061210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7e029-e80f-4ada-a790-b1beb38ebb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:36.079772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5be7e029-e80f-4ada-a790-b1beb38ebb29 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:36.080973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5be7e029-e80f-4ada-a790-b1beb38ebb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:36.148391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 575/1168] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:36.418660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691be1a-7e28-421e-9455-2b702ecf8a78 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:36.470531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:36.529700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691be1a-7e28-421e-9455-2b702ecf8a78 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:36.567512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10041ccf-f249-409f-a5f3-4fb083e3269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:36.568115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041ccf-f249-409f-a5f3-4fb083e3269f None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:36.568392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041ccf-f249-409f-a5f3-4fb083e3269f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:36.568658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041ccf-f249-409f-a5f3-4fb083e3269f None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:36.569329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10041ccf-f249-409f-a5f3-4fb083e3269f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:36.570678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10041ccf-f249-409f-a5f3-4fb083e3269f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:36.590333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 594/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Sat Sep 24 19:33:36 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:33:38.817295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a9e9-39ff-4a34-985a-7cdba5f5b2de None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:38.817862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a9e9-39ff-4a34-985a-7cdba5f5b2de None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:38.817862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a9e9-39ff-4a34-985a-7cdba5f5b2de None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:38.819176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 576/1170] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:38.824674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:38.825149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:38.825258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:38.843595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:38.844867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:38.912956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 595/1171] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:39.187500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a9e9-39ff-4a34-985a-7cdba5f5b2de None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:39.217105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:39.292784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a9e9-39ff-4a34-985a-7cdba5f5b2de None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:39.330006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd356759-fc12-486c-baf2-aabcb93cce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:39.330630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd356759-fc12-486c-baf2-aabcb93cce9e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:39.330902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd356759-fc12-486c-baf2-aabcb93cce9e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:39.331169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd356759-fc12-486c-baf2-aabcb93cce9e None admin] PATH_INFO: `/v3/domains/071c6b8758b243bfb9e83a58aba01d19` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:39.342443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd356759-fc12-486c-baf2-aabcb93cce9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=071c6b8758b243bfb9e83a58aba01d19)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:39.343927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd356759-fc12-486c-baf2-aabcb93cce9e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:39.345570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 577/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Sat Sep 24 19:33:39 2022] GET /identity/v3/domains/071c6b8758b243bfb9e83a58aba01d19 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:39.351678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:39.384549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:39.461197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45770c-75e7-4db5-8d51-dc349273e85b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:39.498139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1d825-66be-4a3c-8f1e-e46d41f1c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:39.498750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1d825-66be-4a3c-8f1e-e46d41f1c331 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:39.499025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1d825-66be-4a3c-8f1e-e46d41f1c331 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:39.499292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1d825-66be-4a3c-8f1e-e46d41f1c331 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:39.499976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1d825-66be-4a3c-8f1e-e46d41f1c331 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:39.501406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1d825-66be-4a3c-8f1e-e46d41f1c331 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:39.531801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 596/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:33:39 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:33:41.781479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8655d9-c567-440c-870c-3ea9028027b7 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:41.782048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8655d9-c567-440c-870c-3ea9028027b7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:41.782048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8655d9-c567-440c-870c-3ea9028027b7 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:41.783353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 578/1174] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:41.788827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:41.789296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:41.789404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:41.808855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:41.810045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:41.877900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 597/1175] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:42.149378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8655d9-c567-440c-870c-3ea9028027b7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:42.251882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:42.268595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8655d9-c567-440c-870c-3ea9028027b7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:42.302670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c326390-5573-4e7b-8060-1b1ba1848819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:42.303302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c326390-5573-4e7b-8060-1b1ba1848819 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:42.303608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c326390-5573-4e7b-8060-1b1ba1848819 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:42.303881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c326390-5573-4e7b-8060-1b1ba1848819 None admin] PATH_INFO: `/v3/domains/071c6b8758b243bfb9e83a58aba01d19` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:42.305265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c326390-5573-4e7b-8060-1b1ba1848819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=071c6b8758b243bfb9e83a58aba01d19)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:42.307329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c326390-5573-4e7b-8060-1b1ba1848819 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:42.308965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 579/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Sat Sep 24 19:33:42 2022] GET /identity/v3/domains/071c6b8758b243bfb9e83a58aba01d19 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:42.314776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:42.330547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:42.416906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27bcc-da71-41e5-b504-530b3628f171 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:42.450820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910957a7-ba0f-41dc-9610-b6936be9f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:42.451452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910957a7-ba0f-41dc-9610-b6936be9f6ec None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:42.451724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910957a7-ba0f-41dc-9610-b6936be9f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:42.451989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910957a7-ba0f-41dc-9610-b6936be9f6ec None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:42.452633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910957a7-ba0f-41dc-9610-b6936be9f6ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:42.454048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910957a7-ba0f-41dc-9610-b6936be9f6ec None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:42.476124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 598/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:33:42 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:33:44.716698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd76c-007f-41a2-8514-b1cd6bbf8daf None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:44.717314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd76c-007f-41a2-8514-b1cd6bbf8daf None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:44.717314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd76c-007f-41a2-8514-b1cd6bbf8daf None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:44.718588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 580/1178] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:44.724178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:44.724927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:44.725104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:44.744078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:44.745335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:44.810114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:44.816407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 599/1179] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:45.091770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd76c-007f-41a2-8514-b1cd6bbf8daf None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:45.194401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd76c-007f-41a2-8514-b1cd6bbf8daf None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:45.229102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:45.229730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:45.230007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:45.230273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:45.240845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:33:45.246243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:45.247493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:33:45.248776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 581/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:45.253934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:45.365221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a33cb-4fb7-4b65-ad0e-df1e5f41deec None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:45.403768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:45.404425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:45.404707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:45.404980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:45.405645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:45.405950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:45.407324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:45.415512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 600/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:45.422109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:45.441052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:45.528542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6d99-0f60-44e5-ad25-938eb988ddf1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:45.563356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:45.564018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:45.564299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:45.564572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] PATH_INFO: `/v3/users/d036cec818e240569efac364d06b378b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:45.579701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] RBAC: Authorizing `identity:get_user(user_id=d036cec818e240569efac364d06b378b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:45.581170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:45.582849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 582/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:45 2022] GET /identity/v3/users/d036cec818e240569efac364d06b378b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:45.589834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:45.655961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:45.695251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e3a32-8d15-40f2-b65a-442f794a0eb3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:45.729836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:45.730536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:45.730700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:45.730966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:45.803453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:45.805196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:45.866333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:45.907033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 601/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:45.913301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:45.988341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:46.012584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c811840-1c37-4b07-bb23-69d24c684af0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:46.047445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7186a-1734-4a3d-9287-43dc9e116cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:46.048088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7186a-1734-4a3d-9287-43dc9e116cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:46.048362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7186a-1734-4a3d-9287-43dc9e116cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:46.048630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7186a-1734-4a3d-9287-43dc9e116cf3 None admin] PATH_INFO: `/v3/projects/b2ce5e6e244d4be5974663bf4359f254` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:46.049749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7186a-1734-4a3d-9287-43dc9e116cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=b2ce5e6e244d4be5974663bf4359f254)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:46.051183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7186a-1734-4a3d-9287-43dc9e116cf3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:46.052859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 583/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:45 2022] GET /identity/v3/projects/b2ce5e6e244d4be5974663bf4359f254 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:46.059556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:46.158806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb552c-0057-4c67-9c34-0de11b86e728 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:46.193698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:46.194297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:46.194572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:46.194840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:46.195601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=b2ce5e6e244d4be5974663bf4359f254, user.id=d036cec818e240569efac364d06b378b) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:46.195909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:46.197862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53a5bf-832b-4b7e-9c47-5d324c8ffca8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:46.203999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 602/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:46 2022] GET /identity/v3/role_assignments?user.id=d036cec818e240569efac364d06b378b&scope.project.id=b2ce5e6e244d4be5974663bf4359f254&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:48.348669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb87807-7b88-4461-a5dd-b756a87bd823 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:48.349201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb87807-7b88-4461-a5dd-b756a87bd823 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:48.349201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb87807-7b88-4461-a5dd-b756a87bd823 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:48.350467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 584/1186] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:48.356450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:48.356911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:48.357010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:48.375557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:48.376752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:48.396955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:48.444265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 603/1187] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:48.707607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb87807-7b88-4461-a5dd-b756a87bd823 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:48.809377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb87807-7b88-4461-a5dd-b756a87bd823 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:48.843576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:48.844202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:48.844475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:48.844735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:48.855560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:33:48.860307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:48.861411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:33:48.862261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 585/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:48.867483 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:48.968373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11088a9-363f-46b4-9b9c-2143f0874273 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:49.002321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:49.002930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:49.003206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:49.003498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:49.004158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:49.004457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:49.005830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:49.015195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 604/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:33:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:49.021204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:49.024776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:49.126771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79748a-2545-4e1c-8cc5-3d1cf4b94f9e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:49.161615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:49.162245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:49.162520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:49.162784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] PATH_INFO: `/v3/users/d036cec818e240569efac364d06b378b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:49.163941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d036cec818e240569efac364d06b378b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:49.165960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:49.167709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 586/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:49 2022] GET /identity/v3/users/d036cec818e240569efac364d06b378b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:49.173778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:49.194985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:49.275973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de14485-bdf5-4386-bd26-9180d8375d65 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:49.309136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b87e18-29e3-4c41-8a13-70376891f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:49.309755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b87e18-29e3-4c41-8a13-70376891f866 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:49.310027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b87e18-29e3-4c41-8a13-70376891f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:49.310293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b87e18-29e3-4c41-8a13-70376891f866 None admin] PATH_INFO: `/v3/projects/b2ce5e6e244d4be5974663bf4359f254` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:49.311390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b87e18-29e3-4c41-8a13-70376891f866 None admin] RBAC: Authorizing `identity:get_project(project_id=b2ce5e6e244d4be5974663bf4359f254)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:49.312862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b87e18-29e3-4c41-8a13-70376891f866 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:49.314544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 605/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:49 2022] GET /identity/v3/projects/b2ce5e6e244d4be5974663bf4359f254 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:49.320792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:49.390961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:49.423247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dd527-bca4-47ca-b59b-a50031d126e3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:49.457727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be8e8c-7889-4d1a-9cae-787ba65ca3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:49.458392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be8e8c-7889-4d1a-9cae-787ba65ca3c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:49.458668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be8e8c-7889-4d1a-9cae-787ba65ca3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:49.458942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be8e8c-7889-4d1a-9cae-787ba65ca3c1 None admin] PATH_INFO: `/v3/projects/b2ce5e6e244d4be5974663bf4359f254/users/d036cec818e240569efac364d06b378b/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:49.460855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be8e8c-7889-4d1a-9cae-787ba65ca3c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2ce5e6e244d4be5974663bf4359f254, user_id=d036cec818e240569efac364d06b378b, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:49.462321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be8e8c-7889-4d1a-9cae-787ba65ca3c1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:49.474993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 587/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:33:49 2022] PUT /identity/v3/projects/b2ce5e6e244d4be5974663bf4359f254/users/d036cec818e240569efac364d06b378b/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:33:51.669861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b38d-fd3e-498a-b53e-1a6fc4469c01 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:51.670425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b38d-fd3e-498a-b53e-1a6fc4469c01 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:51.670425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b38d-fd3e-498a-b53e-1a6fc4469c01 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:51.671778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 606/1193] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:33:51.677190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:51.677617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:51.677726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:51.697109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:33:51.698315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:33:51.771161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 588/1194] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:33:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:33:52.041731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b38d-fd3e-498a-b53e-1a6fc4469c01 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:52.128515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:52.145015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b38d-fd3e-498a-b53e-1a6fc4469c01 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:52.179865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:52.180492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:52.180770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:52.181142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:52.192002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:33:52.197404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:33:52.198520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:33:52.199739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 607/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:33:52.204824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:52.268682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:52.314778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcadf-f688-4125-a9e0-100c38b2199c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:52.349750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:52.350361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:52.350637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:52.350902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:52.351590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:52.351892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:52.353322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:52.361364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 589/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:52.367326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:52.467017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b52aa-13a6-4018-868f-d96196112533 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:52.500727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:52.501344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:52.501618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:52.501883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] PATH_INFO: `/v3/users/d036cec818e240569efac364d06b378b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:52.502989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] RBAC: Authorizing `identity:get_user(user_id=d036cec818e240569efac364d06b378b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:52.504440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:52.506018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 608/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Sat Sep 24 19:33:52 2022] GET /identity/v3/users/d036cec818e240569efac364d06b378b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:52.512320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:52.611749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9200d-9859-45d5-bdf6-630bdb4b6269 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:52.633676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:52.645594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:52.646207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:52.646484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:52.650125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:52.715653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:52.717117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:52.833987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 590/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:33:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:33:52.840461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:52.857144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:52.940700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e4358-6fc5-4836-8a41-94b2004cc5bc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:52.975039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823ade3-89d8-4ee6-8a40-0e09e0ced279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:52.975695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823ade3-89d8-4ee6-8a40-0e09e0ced279 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:52.975973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823ade3-89d8-4ee6-8a40-0e09e0ced279 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:52.976242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823ade3-89d8-4ee6-8a40-0e09e0ced279 None admin] PATH_INFO: `/v3/projects/b2ce5e6e244d4be5974663bf4359f254` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:52.977357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823ade3-89d8-4ee6-8a40-0e09e0ced279 None admin] RBAC: Authorizing `identity:get_project(project_id=b2ce5e6e244d4be5974663bf4359f254)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:52.979358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823ade3-89d8-4ee6-8a40-0e09e0ced279 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:52.981014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 609/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Sat Sep 24 19:33:52 2022] GET /identity/v3/projects/b2ce5e6e244d4be5974663bf4359f254 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:33:52.987208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:33:53.011241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:33:53.090203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7126aab-e762-46f6-9dca-ca3b090e6cf0 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:33:53.123682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:33:53.124290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:33:53.124566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:33:53.124858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:33:53.125576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=b2ce5e6e244d4be5974663bf4359f254, user.id=d036cec818e240569efac364d06b378b) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:33:53.126006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:33:53.127342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962af12-e707-4348-8990-59d2e21efeaa None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:33:53.133804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 591/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:33:52 2022] GET /identity/v3/role_assignments?user.id=d036cec818e240569efac364d06b378b&scope.project.id=b2ce5e6e244d4be5974663bf4359f254&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:02.971059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=59 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:03.130892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=57 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:49.909672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb360-ca92-4ff9-a4bf-6317535b2a5c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:49.910352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb360-ca92-4ff9-a4bf-6317535b2a5c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:49.910352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb360-ca92-4ff9-a4bf-6317535b2a5c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:49.911741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 610/1201] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:34:49.917516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:49.921809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:49.921809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:49.937546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:34:49.938831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:34:50.020324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 592/1202] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:34:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:34:50.301617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fb360-ca92-4ff9-a4bf-6317535b2a5c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:50.425227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fb360-ca92-4ff9-a4bf-6317535b2a5c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:50.471027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:50.471752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:50.472031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:50.472299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:50.487256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:50.488855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:34:50.505881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:50.507289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:34:50.509099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:34:50.509099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:50.509099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:34:50.513323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 611/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:34:50.514647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:50.651794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100d291-8bec-436c-a0c9-aa128b6c0d81 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:50.679206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:50.690394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:50.691009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:50.691310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:50.691656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:50.692360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:50.692660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:50.694135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4be5f-b632-432c-88d8-e63cb0cbcdb2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:50.715789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 593/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:34:50.717368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:50.774982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:50.839075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112750ea-d96d-44df-b6aa-97db343626fb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:50.881351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf910d-b50c-4498-aa2b-246255d738e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:50.881976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf910d-b50c-4498-aa2b-246255d738e4 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:50.882244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf910d-b50c-4498-aa2b-246255d738e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:50.882519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf910d-b50c-4498-aa2b-246255d738e4 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:50.883350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf910d-b50c-4498-aa2b-246255d738e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:50.887234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf910d-b50c-4498-aa2b-246255d738e4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:50.923559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 612/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Sat Sep 24 19:34:50 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:34:53.236235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702a508-2948-4fc9-bf04-1736fe07784e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:53.236803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702a508-2948-4fc9-bf04-1736fe07784e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:53.236982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702a508-2948-4fc9-bf04-1736fe07784e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:53.238851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 594/1206] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:34:53.244281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:53.244745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:53.244831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:53.264315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:34:53.265560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:34:53.347298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 613/1207] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:34:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:34:53.658467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702a508-2948-4fc9-bf04-1736fe07784e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:53.779654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702a508-2948-4fc9-bf04-1736fe07784e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:53.819748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:53.822629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:53.822629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:53.822629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:53.837328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:34:53.843611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:53.844909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:34:53.845786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 595/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:34:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:53.850946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:53.899604 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:53.980270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fabb7-c030-4c5c-9bf6-4abbbad3de8f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:54.023656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:54.024418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:54.024694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:54.024988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:54.025835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:54.026135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:54.027528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:54.038552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 614/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:34:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:54.044872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:54.116587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:54.189494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb3415-8a04-4f8e-b595-e1e63badec39 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:54.237707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:54.238358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:54.238767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:54.239136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:54.252604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:54.254509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:34:54.271420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:54.272586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:34:54.274296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:34:54.274296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:54.274296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:34:54.275966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 596/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:54.281865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:54.399173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:54.402107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42cc71-c5e1-4906-b1c4-debdc4e6bd2f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:54.442646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:54.443283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:54.443663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:54.443861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:54.444505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:54.444808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:54.446987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:54.459528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 615/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:54.467507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:54.582926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c729f2-d54c-4035-a780-22c611674bf4 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:54.584597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:54.620184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:54.620789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:54.621172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:54.621403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:54.629273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:54.630758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:34:54.643009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:54.644324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:34:54.645314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:34:54.650228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:54.650631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 597/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:34:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:54.762303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d9662-b5e6-4bff-876c-346e8fba6b2d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:54.799883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:54.800435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:54.800709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:54.800973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:54.801722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:54.802050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:54.803537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:54.823963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 616/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Sat Sep 24 19:34:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:54.829786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:54.953755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764cf2-3a7f-4850-b1e3-aff0b2dca378 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:54.992037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:54.993017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:54.993307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:54.993583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:55.028653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:55.080275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:55.081751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:55.204723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 598/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:34:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:34:55.214065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:55.237455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:55.344493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904adb11-a0a7-4a6b-9ae9-6375019cef0d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:55.385986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:55.386792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:55.387076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:55.387357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:55.401588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:55.404157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:34:55.420830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:55.421878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:34:55.423543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:34:55.423543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:55.423543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:34:55.423543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 617/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:34:55.430472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:55.513712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:55.561623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85725e88-2320-4425-96b9-6c88b27ec674 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:55.599417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:55.600075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:55.600340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:55.600669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:55.601453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:55.601737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:55.603111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:55.620406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 599/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:55.622145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:55.702675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:55.746482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6ecd-afb9-4d4f-8b60-5625d1b6a9e3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:55.796614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:55.798033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:55.798536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:55.798951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:55.807282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:55.808337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:34:55.821088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:55.822375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:34:55.823318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 618/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:34:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:55.828692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:55.953951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1148-0f01-4bc9-abd8-318b0e8b100e None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:55.981698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:55.995607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:55.996627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:55.997061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:55.997531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:55.999091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:55.999731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:56.001347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8cdd6-1718-4e5c-9498-321ae8d7dd80 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:56.016544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 600/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:34:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:56.022634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:56.151775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0f474-365e-433c-8dc6-c1b6d31e4586 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:56.189739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:56.197081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:56.197720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:56.197974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:56.198248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:56.198979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=3a2247b0344a416788f593199de5e42c) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:56.199286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:56.201252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deff26-60c2-4d24-811b-b26b9b3bbff1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:56.208116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 619/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:34:56 2022] GET /identity/v3/role_assignments?user.id=3a2247b0344a416788f593199de5e42c&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:58.606835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb4fb-73b3-4142-984f-2ff31e164e93 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:58.607481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb4fb-73b3-4142-984f-2ff31e164e93 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:58.607481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb4fb-73b3-4142-984f-2ff31e164e93 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:58.609036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 601/1220] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:34:58.614792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:58.615254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:58.615357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:58.636412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:34:58.637802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:34:58.719173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 620/1221] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:34:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:34:58.991562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb4fb-73b3-4142-984f-2ff31e164e93 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:59.112941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb4fb-73b3-4142-984f-2ff31e164e93 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:59.154656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:59.155314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:59.155620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:59.155888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:59.167612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:34:59.173449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:59.174687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:34:59.175773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 602/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:34:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:59.181454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:59.285066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aef08-f386-4c46-af91-8fa3ee681913 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:59.310717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:59.326114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:59.326771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:59.327009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:59.327295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:59.328115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:59.328327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:59.329698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:59.339008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 621/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:34:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:34:59.345184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:59.396773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:59.463642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a0260-d1d2-410c-be97-b9fbf1db45d7 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:59.502131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:59.503130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:59.503522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:59.503866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:59.514707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:59.516204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:34:59.531498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:59.533246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:34:59.534474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:34:59.534474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:34:59.534474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:34:59.534474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 603/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:59.539552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:59.646091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f946ec-dc6b-4e60-92c1-a72673fb03eb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:59.683217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:59.684028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:59.684362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:59.684698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:59.685349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:34:59.685651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:59.687017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:59.700401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 622/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:34:59.706817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:59.819969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:34:59.829938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef6095-b360-40fc-b313-c452b515ea26 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:34:59.866790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:34:59.867446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:34:59.867819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:34:59.868087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:34:59.875985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:34:59.877507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:34:59.900178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:34:59.901514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:34:59.902670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 604/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:34:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:34:59.907352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:34:59.959776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:00.018368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab4118-f146-428a-b1c9-f412e6babb6b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:00.056696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:00.057328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:00.057606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:00.057881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:00.058954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:00.058954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:00.060737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:00.083767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 623/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Sat Sep 24 19:34:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:00.087502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:00.195789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805993d-cbcc-40d6-bab2-0482e9fd9897 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:00.233941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:00.234581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:00.234863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:00.235133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:00.247324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:00.248886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:00.264222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:00.265425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:00.266375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:00.266375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:00.266375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:00.266375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 605/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:00.276022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:00.384357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e617e-7140-4529-8558-ea82c6c5230e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:00.387953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:00.425648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:00.426661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:00.427134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:00.427636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:00.428928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:00.430329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:00.432995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:00.449730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 624/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:00.455500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:00.467020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:00.577960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88538c2e-0480-4bba-9938-7de15f30cf8b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:00.611812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:00.612457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:00.612726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:00.612999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:00.620673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:00.622096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:35:00.634206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:00.635373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:00.636528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 606/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:35:00.641493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:00.756600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79f30-42a0-41c1-8f33-ce89a1fd0852 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:00.797102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:00.797712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:00.797995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:00.798297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:00.799001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:00.799342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:00.800932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326409af-d617-4832-ad74-bc2bf2742706 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:00.824688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 625/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:00.825999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:00.895744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:00.940069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9b01-3bb0-479c-9890-b51598b89ed3 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:00.980947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc4e32-ec68-4188-add3-bc234a7950c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:00.981606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc4e32-ec68-4188-add3-bc234a7950c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:00.981882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc4e32-ec68-4188-add3-bc234a7950c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:00.982147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc4e32-ec68-4188-add3-bc234a7950c0 None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/3a2247b0344a416788f593199de5e42c/roles/023b51b34daf482db7bc23d3553be224` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:00.999407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc4e32-ec68-4188-add3-bc234a7950c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=3a2247b0344a416788f593199de5e42c, role_id=023b51b34daf482db7bc23d3553be224)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:01.001286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc4e32-ec68-4188-add3-bc234a7950c0 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:01.016438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 607/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:35:00 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/3a2247b0344a416788f593199de5e42c/roles/023b51b34daf482db7bc23d3553be224 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:35:03.262617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7427-3ce7-462c-aa52-01f27f3c23fd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:03.263213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7427-3ce7-462c-aa52-01f27f3c23fd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:03.263325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7427-3ce7-462c-aa52-01f27f3c23fd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:03.264934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 626/1233] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:03.270590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:03.271033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:03.271189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:03.291727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:03.293108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:03.370572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 608/1234] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:03.655661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc7427-3ce7-462c-aa52-01f27f3c23fd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:03.683277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:03.782907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc7427-3ce7-462c-aa52-01f27f3c23fd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:03.820946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:03.821643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:03.821949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:03.822224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:03.833112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:35:03.838264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:03.839249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:03.840596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 627/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Sat Sep 24 19:35:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:03.845557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:03.893832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:03.954123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bbcd3-6a68-4e0f-b9e6-f7cdef0c1be3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:03.998743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:03.999621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:03.999934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:04.000276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:04.001017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] RBAC: Adding query filter params (name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:04.001382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:04.002986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:04.014711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 609/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Sat Sep 24 19:35:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:04.020712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:04.118153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:04.136518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70809e-d0ff-4c87-ab51-16e8bd44d23f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:04.173902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:04.174518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:04.174817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:04.175123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:04.186132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:04.187609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:04.204506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:04.205729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:04.206641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:04.206641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:04.206641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:04.206641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 628/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:04.212962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:04.328694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a20ce-0dbc-4758-8e6a-e046589a4433 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:04.339454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:04.367034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:04.367680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:04.367961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:04.368231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:04.368869 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:04.369173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:04.370552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:04.383081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 610/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:04.389080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:04.501332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c61f22-f019-4904-b1ab-da74c2a8faba None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:04.539549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:04.540694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:04.540969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:04.541318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:04.546430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:04.550134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:04.551822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:35:04.564717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:04.565865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:04.572774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 629/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:04.573373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:04.685626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ecc51-a426-429d-b901-62d0e061a96f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:04.722889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:04.723667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:04.723887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:04.724152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:04.724981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:04.725283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:04.726803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:04.747506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 611/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Sat Sep 24 19:35:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:04.753542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:04.868639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a839b7-8127-4fdb-b072-fb4b424b2a60 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:04.904463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:04.905121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:04.905396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:04.905675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:04.975816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:04.983383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:04.985007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:05.098538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 630/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:35:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:35:05.099932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:05.134536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:05.216166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803ea52-758a-4ca4-95fb-3276a773ca72 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:05.253949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:05.254588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:05.254868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:05.255144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:05.266172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:05.267653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:05.284904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:05.285981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:05.287278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:05.287278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:05.287278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:05.287278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 612/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:05.292269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:05.394591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:05.400716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfde4ce-140a-48ff-bce8-beec605f1e46 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:05.439143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:05.439871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:05.440151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:05.440453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:05.441117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:05.441415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:05.442858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:05.455061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 631/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:05.461023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:05.544451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:05.569286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c701f2b-aa42-43f6-a4d1-34b05a570d7f None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:05.608833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:05.609478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:05.609757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:05.610026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:05.618611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:05.620217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:35:05.632116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:05.633370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:05.634478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 613/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:05.639400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:05.750411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13823d-f2b6-436f-81df-ebaba7a3d0b7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:05.800525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:05.801157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:05.801425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:05.801689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:05.802384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:05.802686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:05.804222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac1751-7b1c-4465-9a3e-512f880734bb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:05.817966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 632/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:35:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:05.825135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:05.943556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b7aa2-0944-430b-9d5b-671430034d42 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:05.986941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:05.987601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:05.987877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:05.988142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:05.988917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] RBAC: Adding query filter params (role.id=023b51b34daf482db7bc23d3553be224, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=3a2247b0344a416788f593199de5e42c) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:05.989549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:05.991159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee9951-b9b1-4270-b6f0-085301cb6949 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:05.998584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 614/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:35:05 2022] GET /identity/v3/role_assignments?user.id=3a2247b0344a416788f593199de5e42c&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=023b51b34daf482db7bc23d3553be224 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:08.583624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97310-6713-4d8e-9108-6fdee239d8bc None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:08.584185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97310-6713-4d8e-9108-6fdee239d8bc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:08.584365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97310-6713-4d8e-9108-6fdee239d8bc None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:08.586243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 633/1247] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:08.592738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:08.593238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:08.593361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:08.599707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:08.613356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:08.614546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:08.684705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 615/1248] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 19:35:08.954387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97310-6713-4d8e-9108-6fdee239d8bc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:08.974068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:09.072844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97310-6713-4d8e-9108-6fdee239d8bc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:09.109080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:09.109708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:09.109987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:09.110260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:09.122726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:35:09.127774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:09.128970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:09.130435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 634/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:35:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:09.135903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:09.245787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb16266-8323-40c9-a78b-ebae560472c7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:09.277771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:09.283549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:09.284231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:09.284571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:09.284847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:09.285640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:09.285951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:09.287500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:09.297102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 616/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:35:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:09.303372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:09.390684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:09.417815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91709-5144-4a85-baef-777c933b2e02 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:09.457694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:09.458323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:09.458603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:09.458872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:09.471328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:09.472995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:09.488295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:09.489494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:09.491014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:09.491014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:09.491014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:09.491014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 635/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:09.496281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:09.616086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b568dc0-1991-49cf-944f-26086a8fd4de None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:09.661632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:09.662366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:09.662709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:09.662978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:09.663977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:09.664090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:09.665614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:09.683484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 617/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:09.686344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:09.817784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf4be6-053c-47cc-99c6-f060895b673e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:09.849051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:09.857584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:09.858224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:09.858499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:09.858765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:09.868005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:09.869479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:35:09.881669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:09.883301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:09.884572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 636/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:09.889201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:09.949022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:09.998090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84b42a-2358-4647-b093-ab90aefbe310 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:10.036102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:10.036728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:10.036989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:10.037254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:10.037996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:10.038295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:10.039864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:10.061638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 618/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Sat Sep 24 19:35:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:10.067552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:10.171224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5657fc-434b-416c-951e-f778776d47f5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:10.210341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:10.211936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:10.212300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:10.212590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:10.271590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:10.287787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:10.289868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:10.388750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 637/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:35:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:35:10.394640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:10.491457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beaa7b-690e-44fb-8bdb-f48670f0b921 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:10.503849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:10.524964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:10.525570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:10.525843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:10.526108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:10.535768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:10.537200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:10.551265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:10.552396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:10.553475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:10.553475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:10.553475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:10.553475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 619/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:10.559024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:10.644192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:10.656657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea17c6-a939-44d3-89de-b70b42f4ef9f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:10.690462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:10.691070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:10.691345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:10.691643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:10.692272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:10.692569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:10.693918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:10.704797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 638/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:10.710432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:10.816140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fea81-581b-4c0f-a359-379563b5662d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:10.849342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:10.849965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:10.850243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:10.850514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:10.857742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:10.859190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:35:10.871044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:10.872140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:10.873250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 620/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:10.878123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:10.977061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b83b07-2225-4cb2-9399-3dd108ea8e99 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:11.009974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:11.010569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:11.010855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:11.011125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:11.011834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:11.012143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:11.013559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce75e-c4eb-49e7-a052-e21225ee4884 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:11.025898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 639/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:11.033090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:11.060413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:11.133094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df2d43-bd1d-4859-a99a-7b4b8446ee73 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:11.167064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:11.167923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:11.168385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:11.168779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:11.169509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=3a2247b0344a416788f593199de5e42c) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:11.169814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:11.171254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55ed8-c337-4009-8407-5315eec18c68 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:11.177111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 621/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:35:11 2022] GET /identity/v3/role_assignments?user.id=3a2247b0344a416788f593199de5e42c&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:13.316929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46713b83-78f8-4adb-a57e-accfad5033d4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:13.317440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46713b83-78f8-4adb-a57e-accfad5033d4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:13.317440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46713b83-78f8-4adb-a57e-accfad5033d4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:13.318704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 640/1261] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:13.324478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:13.324885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:13.325024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:13.342676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:13.343902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:13.409148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 622/1262] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:13.681089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46713b83-78f8-4adb-a57e-accfad5033d4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:13.692140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:13.793729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46713b83-78f8-4adb-a57e-accfad5033d4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:13.838259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:13.838992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:13.839413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:13.839785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:13.850901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:35:13.857296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:13.858528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:13.859616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 641/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:35:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 19:35:13.864672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:13.884074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:13.964953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05264843-69a6-44f5-b7c7-108522fe4519 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:13.998605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:13.999212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:13.999515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:13.999790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.000448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:14.000749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.002119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.010263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 623/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:35:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:14.015970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:14.084627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:14.119245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbb0d5-0b5e-446f-beb1-cbbc5c9f73bb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:14.151653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:14.152268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:14.152544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:14.152811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.162920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.164989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:14.178701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:14.179704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:14.180926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:14.180926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:14.180926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:14.180926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 642/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:14.186726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:14.267362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:14.296431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e7f22-c4d0-4326-a5e3-0f57567c5675 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:14.331792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:14.332716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:14.332716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:14.332994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.333520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:14.333820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.335190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.347398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 624/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:14.354580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:14.453153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd8874-d984-4eed-b1d0-e378882d5688 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:14.471886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:14.486261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:14.486877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:14.487150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:14.487415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.495237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.496694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:35:14.507414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:14.508505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:14.509697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 643/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:14.514384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:14.610734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea86f1b-d3f5-49bd-a1bd-95eb093b96ab None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:14.643222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:14.643835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:14.644108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:14.644380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.645088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:14.645389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.646801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.655134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:14.663821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 625/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:14.670187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:14.773001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2afc15-86a4-4b08-aaf5-43d9d3f686f1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:14.807142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:14.807787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:14.808063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:14.808330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.818430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.819908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:14.833555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:14.834684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:14.835824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:14.835824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:14.835824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:14.835824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 644/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:14.841681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:14.939534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ec398-18be-41cb-91b2-6861bc598178 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:14.972241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:14.972824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:14.973098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:14.973363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:14.973989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:14.974283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:14.975662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:14.988022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 626/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:14.994399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:15.025719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:15.097283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd20fb3-ccf7-43e4-8a74-a1735cad13c0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:15.129581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:15.130203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:15.130481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:15.130753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:15.137755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:15.139189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:35:15.151755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:15.152859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:15.153967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 645/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:35:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:15.158734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:15.217970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:15.259796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3916a-de20-432d-a467-9915d78a93b6 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:15.293257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:15.293860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:15.294140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:15.294415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:15.295096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:15.295402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:15.296873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd375c-32fe-4e0d-9a51-f0e4db10c284 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:15.308986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 627/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:35:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:15.315070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:15.404380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:15.413203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e420a-47ca-4668-ab6d-c4ca2198fa72 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:15.447367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e92aa1-7a58-4657-969c-cf4296e83535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:15.448044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e92aa1-7a58-4657-969c-cf4296e83535 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:15.448324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e92aa1-7a58-4657-969c-cf4296e83535 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:15.448594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e92aa1-7a58-4657-969c-cf4296e83535 None admin] PATH_INFO: `/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/3a2247b0344a416788f593199de5e42c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:15.450484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e92aa1-7a58-4657-969c-cf4296e83535 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca32d28730cf460ba7618a86aef5a4f0, user_id=3a2247b0344a416788f593199de5e42c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:15.451951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e92aa1-7a58-4657-969c-cf4296e83535 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:15.464366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 646/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Sat Sep 24 19:35:15 2022] PUT /identity/v3/projects/ca32d28730cf460ba7618a86aef5a4f0/users/3a2247b0344a416788f593199de5e42c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:35:17.622998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca7b-2522-4c9f-ab16-660cde15efad None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:17.623601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca7b-2522-4c9f-ab16-660cde15efad None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:17.623601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca7b-2522-4c9f-ab16-660cde15efad None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:17.624923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 628/1274] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:17.630225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:17.630634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:17.630766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:17.650047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:17.651225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:17.715790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 647/1275] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:17.989651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1ca7b-2522-4c9f-ab16-660cde15efad None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:18.100764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1ca7b-2522-4c9f-ab16-660cde15efad None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:18.119540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:18.136271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:18.136894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:18.137172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:18.137440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:18.147270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 19:35:18.152227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:18.153338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:18.154366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 629/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Sat Sep 24 19:35:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:18.159285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:18.237634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:18.256294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385d19-5ae7-4d9b-8761-74f0dea1527c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:18.288522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:18.289749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:18.290027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:18.290298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:18.290951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:18.291249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:18.292636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:18.301870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 648/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Sat Sep 24 19:35:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:18.308557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:18.404614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75490f7-4a25-49ca-86a6-345bb3537141 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:18.438094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:18.438713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:18.438987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:18.439251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:18.448820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:18.450254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:18.464709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:18.465872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:18.467006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:18.467006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:18.467006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:18.467577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 630/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:18.471467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:18.568728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ca7c6-9a2d-4da1-bb2a-2fc448f9a77c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:18.600734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:18.601315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:18.601637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:18.601906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:18.602534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:18.603317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:18.604693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:18.615969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 649/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:18.621574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:18.651960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:18.721158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f3738-3019-4103-a724-f25ad5ca482b None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:18.754445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:18.755060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:18.755334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:18.755624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:18.762803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:18.764256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 19:35:18.775591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:18.776674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:18.777653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 631/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:18.782479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:18.786235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:18.879145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7d1b5-3028-40f3-b160-a06b52cc9a61 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:18.912029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:18.912612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:18.912883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:18.913146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:18.913855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:18.914155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:18.915583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:18.932845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 650/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Sat Sep 24 19:35:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:18.938577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:19.028623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:19.038431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc9b1e-c49a-44d3-82d5-65f56d39b2a2 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:19.072833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:19.073437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:19.073713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:19.073980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:19.139024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:19.140422 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:19.225701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:19.236570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 632/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Sat Sep 24 19:35:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 19:35:19.242388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:19.302564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:19.340284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072a6f3-dab3-432d-8ad8-104fb98a01ef None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:19.373098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:19.373707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:19.373983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:19.374252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:19.384047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:19.385472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 19:35:19.400264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:19.401386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 19:35:19.402482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 19:35:19.402482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 19:35:19.402482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:19.402482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 651/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:19.406962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:19.502641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3b38-8e14-4bb6-b83a-c9d140d0c48d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:19.535308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:19.535929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:19.536207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:19.536479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:19.537109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:19.537410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:19.538777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:19.550313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 633/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Sat Sep 24 19:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:19.556228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:19.655291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ad845-d1cd-4f5d-818b-ada86f7eba38 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:19.668756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:19.689249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:19.689864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:19.690141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:19.690408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:19.697849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:19.699289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 19:35:19.710381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:19.712034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Sep 24 19:35:19.713006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 652/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Sat Sep 24 19:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:19.718181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:19.767257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:19.816544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529cbae-a012-40c9-bf5f-23f77860952d None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:19.849826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:19.850419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:19.850695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:19.850963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:19.851739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:19.852046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:19.853466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f56a4-cc10-4707-8a9d-9634d1c5cb22 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:19.866699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 634/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Sat Sep 24 19:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:19.872736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:19.970402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2db2a-fc1e-4dda-b3a5-01a5068612d9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:20.002557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:20.003140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:20.003415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:20.003707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:20.004418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] RBAC: Adding query filter params (role.id=dff984ec428b4b178510d79cce0a8cb3, scope.project.id=ca32d28730cf460ba7618a86aef5a4f0, user.id=3a2247b0344a416788f593199de5e42c) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:20.004721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:20.006124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb864f-3bc2-4cf4-a908-188f08ede9d9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:20.012230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 653/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Sat Sep 24 19:35:19 2022] GET /identity/v3/role_assignments?user.id=3a2247b0344a416788f593199de5e42c&scope.project.id=ca32d28730cf460ba7618a86aef5a4f0&role.id=dff984ec428b4b178510d79cce0a8cb3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:22.181136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa562c76-7949-4b99-a8ff-7805dca8780c None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:22.181727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa562c76-7949-4b99-a8ff-7805dca8780c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:22.181727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa562c76-7949-4b99-a8ff-7805dca8780c None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:22.183038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 635/1288] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:22.188278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:22.188717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:22.188817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:22.207242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:22.208440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:22.236203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:22.272646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 654/1289] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:22.535226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa562c76-7949-4b99-a8ff-7805dca8780c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:22.634572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa562c76-7949-4b99-a8ff-7805dca8780c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:22.648901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:22.680724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:22.681513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:22.681787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:22.682050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:22.682634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:22.684032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:22.692807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:22.694045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:22.694981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 636/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:35:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:22.700062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:22.807156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc3414-9ef2-460f-8cba-32c39c77c507 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:22.841459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:22.842053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:22.842326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:22.842592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:22.843205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:22.843525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:22.844874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5546061-e8de-4029-89c4-db13de233900 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:22.849485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 655/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Sat Sep 24 19:35:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:22.855911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:22.954454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b57af-b511-4062-a8ae-836b1400a78a None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:22.987777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:22.988371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:22.988646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:22.988912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] PATH_INFO: `/v3/services` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:22.989535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:22.989834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:22.991199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed51da7-4bb3-4a64-bed5-763beb6c94d5 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:22.995684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 637/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Sat Sep 24 19:35:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:25.109143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b65568-95e2-4079-ae08-86f28d2de4cb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:25.109686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b65568-95e2-4079-ae08-86f28d2de4cb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:25.109686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b65568-95e2-4079-ae08-86f28d2de4cb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:25.111104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 656/1293] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:25.116425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e16f80-af01-4f43-b209-25e5240bc427 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:25.116820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e16f80-af01-4f43-b209-25e5240bc427 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:25.117093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e16f80-af01-4f43-b209-25e5240bc427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:25.135671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f3e16f80-af01-4f43-b209-25e5240bc427 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:25.136852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f3e16f80-af01-4f43-b209-25e5240bc427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:25.165750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:25.202447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 638/1294] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:25.470450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b65568-95e2-4079-ae08-86f28d2de4cb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:25.501076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:25.571083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b65568-95e2-4079-ae08-86f28d2de4cb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:25.604216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600dd82-4a93-4e22-aa51-9466ab36cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:25.604808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600dd82-4a93-4e22-aa51-9466ab36cad4 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:25.605084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600dd82-4a93-4e22-aa51-9466ab36cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:25.605353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600dd82-4a93-4e22-aa51-9466ab36cad4 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:25.606003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600dd82-4a93-4e22-aa51-9466ab36cad4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:25.607352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600dd82-4a93-4e22-aa51-9466ab36cad4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:25.620903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 657/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Sat Sep 24 19:35:25 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:35:27.770175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f93127-a1d4-42af-ae2a-507b57ab3ccd None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:27.770843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f93127-a1d4-42af-ae2a-507b57ab3ccd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:27.770843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f93127-a1d4-42af-ae2a-507b57ab3ccd None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:27.772546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 639/1296] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:35:27.778754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:27.779243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:27.779387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:27.799344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:27.800681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:27.875279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 658/1297] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:28.153497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f93127-a1d4-42af-ae2a-507b57ab3ccd None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:28.251692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f93127-a1d4-42af-ae2a-507b57ab3ccd None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:28.284120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:28.284731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:28.285002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:28.285265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:28.285828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:28.287213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:28.295744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:28.296917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:28.297948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 640/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:35:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:28.302716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:28.322042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:28.400790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723741cc-044a-4598-94fe-fe8bef2499f9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:28.434235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:28.434827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:28.435102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:28.435368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:28.436021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:28.436321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:28.437680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:28.442678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 659/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Sat Sep 24 19:35:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:28.449192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:28.480141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:28.552142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14a033-b604-441a-87da-3bb60b6bd1d1 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:28.585698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:28.586973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:28.587249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:28.587548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:28.588238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] RBAC: Adding query filter params (interface=public, service_id=f60ea4c9e3d54840af5f907bd915fdbe, region_id=RegionOne) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:28.588543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:28.589908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfab09-2ebb-4b71-a9a5-b98be6118679 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:28.594742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 641/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Sat Sep 24 19:35:28 2022] GET /identity/v3/endpoints?service_id=f60ea4c9e3d54840af5f907bd915fdbe&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:28.600375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:28.678416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:28.696721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd94bd-6c5f-4260-984b-65fbccf0ce53 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:28.729740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd368e15-375c-43c6-9091-d45b814cae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:28.730328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd368e15-375c-43c6-9091-d45b814cae63 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:28.730603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd368e15-375c-43c6-9091-d45b814cae63 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:28.730869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd368e15-375c-43c6-9091-d45b814cae63 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:28.731487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd368e15-375c-43c6-9091-d45b814cae63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:28.732850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd368e15-375c-43c6-9091-d45b814cae63 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:28.740028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 660/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Sat Sep 24 19:35:28 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:35:30.889635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de85eec-0826-4687-93b4-dc53a7893df0 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:30.890216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de85eec-0826-4687-93b4-dc53a7893df0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:30.890216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de85eec-0826-4687-93b4-dc53a7893df0 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:30.891543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 642/1302] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:35:30.896563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:30.896965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:30.897105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:30.915615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:35:30.916779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:35:30.981958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 661/1303] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:35:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 19:35:31.254601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de85eec-0826-4687-93b4-dc53a7893df0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:31.351496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:31.359738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de85eec-0826-4687-93b4-dc53a7893df0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:31.399599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:31.400228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:31.400506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:31.400776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:31.401349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:31.402719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 19:35:31.411266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 19:35:31.412564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application with Lock( Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self._enter() Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application return self.creator() Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application created_value = creator( Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: ERROR keystone.server.flask.application Sep 24 19:35:31.413609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 643/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Sat Sep 24 19:35:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 19:35:31.418444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:31.491280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:31.514697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1adba7-4632-4a3d-9e08-f86c7e95e9ff None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:31.547591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:31.548178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:31.548452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:31.548716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:31.549336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:35:31.549630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:31.550979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fea629-7a6c-43ef-bf0e-41faddd14c8f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:31.556708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 662/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Sat Sep 24 19:35:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:35:31.562439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:35:31.661604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030a8f-f883-49a5-94e5-b06c07b20a57 None admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:35:31.697700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79fa99-1253-4ddf-8219-7e084810d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:35:31.698301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fa99-1253-4ddf-8219-7e084810d380 None admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:35:31.698572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fa99-1253-4ddf-8219-7e084810d380 None admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:35:31.698834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fa99-1253-4ddf-8219-7e084810d380 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:35:31.699522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79fa99-1253-4ddf-8219-7e084810d380 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:35:31.700879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79fa99-1253-4ddf-8219-7e084810d380 None admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:35:31.720393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 644/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:35:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:35:41.553457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=30 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:35:41.717282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:36.324883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63607-4544-4467-8d3b-32a29febd2f1 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:36.325393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63607-4544-4467-8d3b-32a29febd2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:36.325393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63607-4544-4467-8d3b-32a29febd2f1 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:36.326893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 663/1307] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:36.333555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51548-ae02-4ee0-9eeb-3b7dd26c0e29 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:36.335255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51548-ae02-4ee0-9eeb-3b7dd26c0e29 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:36.336491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51548-ae02-4ee0-9eeb-3b7dd26c0e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:36.474214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-41f51548-ae02-4ee0-9eeb-3b7dd26c0e29 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:36.476165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-41f51548-ae02-4ee0-9eeb-3b7dd26c0e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:36.599585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 645/1308] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:36.893252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1ad0d-808a-474f-8961-dd6209343d3b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:36.893513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1ad0d-808a-474f-8961-dd6209343d3b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:36.893778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1ad0d-808a-474f-8961-dd6209343d3b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:36.895344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 664/1309] 10.222.0.55 () {54 vars in 964 bytes} [Sat Sep 24 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:36.902488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7d925-9382-4ae9-90e7-e2337b1f9b10 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:36.902917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7d925-9382-4ae9-90e7-e2337b1f9b10 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:36.903469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7d925-9382-4ae9-90e7-e2337b1f9b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:36.984641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ecf7d925-9382-4ae9-90e7-e2337b1f9b10 None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:36.986013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ecf7d925-9382-4ae9-90e7-e2337b1f9b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:37.008570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:37.075198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 646/1310] 10.222.0.55 () {58 vars in 1061 bytes} [Sat Sep 24 19:36:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:37.084373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1ad0d-808a-474f-8961-dd6209343d3b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:37.087061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-43e1ad0d-808a-474f-8961-dd6209343d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:37.215684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1ad0d-808a-474f-8961-dd6209343d3b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:37.304393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:37.305183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:37.305621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:37.306005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:37.308160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:37.373490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:37.441877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:37.443585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872658e6-29b7-454e-b6ec-dac4aeb202ad service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:37.467331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 665/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:36:40.010028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59e3ed-5d6a-44de-83ba-a9df028e495c None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:40.010548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59e3ed-5d6a-44de-83ba-a9df028e495c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:40.010548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59e3ed-5d6a-44de-83ba-a9df028e495c None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:40.011853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 647/1312] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:40.017290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb3b66-a2db-4b13-a331-2c28eec6bd52 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:40.017711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb3b66-a2db-4b13-a331-2c28eec6bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:40.017833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb3b66-a2db-4b13-a331-2c28eec6bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:40.105164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a1fb3b66-a2db-4b13-a331-2c28eec6bd52 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:40.106382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a1fb3b66-a2db-4b13-a331-2c28eec6bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:40.204021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 666/1313] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:40.496206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59e3ed-5d6a-44de-83ba-a9df028e495c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:40.497729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7c59e3ed-5d6a-44de-83ba-a9df028e495c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:40.600490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59e3ed-5d6a-44de-83ba-a9df028e495c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:40.670144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:40.672872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:40.673712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:40.674170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:40.674477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:40.676269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:40.783841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:40.785211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1fe3-17f6-4e03-98b5-6b3d8db0a812 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:40.804630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 648/1314] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:36:43.271747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60e94-947a-4c0b-9fdf-93f6406a0b86 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:43.273006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60e94-947a-4c0b-9fdf-93f6406a0b86 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:43.273006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60e94-947a-4c0b-9fdf-93f6406a0b86 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:43.274194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 667/1315] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:43.279485 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfdb2d-c9e0-495a-b45f-b95c01f15f3c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:43.279936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfdb2d-c9e0-495a-b45f-b95c01f15f3c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:43.280042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfdb2d-c9e0-495a-b45f-b95c01f15f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:43.345229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7bdfdb2d-c9e0-495a-b45f-b95c01f15f3c None None] MFA Rules not processed for user `475b2f85e68d43ac91521a15a826e5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:43.346344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7bdfdb2d-c9e0-495a-b45f-b95c01f15f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:43.397725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:43.442484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 649/1316] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:43.725708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60e94-947a-4c0b-9fdf-93f6406a0b86 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:43.727231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-13d60e94-947a-4c0b-9fdf-93f6406a0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:43.736955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:43.830476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60e94-947a-4c0b-9fdf-93f6406a0b86 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:43.902605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:43.903202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:43.903501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:43.903769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:43.905339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:43.968485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:44.017254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:44.018680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ab1ae-4349-40d8-9704-7672c6a8d4a2 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:44.036524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 668/1317] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:36:46.744431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8745f41-8d22-4190-b986-d5c1e13ef5a1 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:46.745108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8745f41-8d22-4190-b986-d5c1e13ef5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:46.745108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8745f41-8d22-4190-b986-d5c1e13ef5a1 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:46.746378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 650/1318] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:36:46.752076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c68b6-5ae4-4dd2-82f6-dc6dd133d1d5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:46.752610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c68b6-5ae4-4dd2-82f6-dc6dd133d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:46.752730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c68b6-5ae4-4dd2-82f6-dc6dd133d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:46.821608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ae5c68b6-5ae4-4dd2-82f6-dc6dd133d1d5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:46.822772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ae5c68b6-5ae4-4dd2-82f6-dc6dd133d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:46.919833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 669/1319] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:36:47.212832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8745f41-8d22-4190-b986-d5c1e13ef5a1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:47.214374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f8745f41-8d22-4190-b986-d5c1e13ef5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:47.316889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8745f41-8d22-4190-b986-d5c1e13ef5a1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:47.388496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:47.389089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:47.389365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:47.389634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:47.391208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:47.398751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:47.498506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:47.499991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324bca6-0711-4991-bdd9-5f09fa224674 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:47.517913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 651/1320] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:36:50.322942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67456c29-be52-41e6-a6e2-1ce8ee888b11 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:50.323491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67456c29-be52-41e6-a6e2-1ce8ee888b11 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:50.323491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67456c29-be52-41e6-a6e2-1ce8ee888b11 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:50.324768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 670/1321] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:50.330000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c915e4-0675-48eb-a92b-054565b80993 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:50.330428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c915e4-0675-48eb-a92b-054565b80993 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:50.330523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c915e4-0675-48eb-a92b-054565b80993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:50.397563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b3c915e4-0675-48eb-a92b-054565b80993 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:50.398804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b3c915e4-0675-48eb-a92b-054565b80993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:50.463684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:50.506835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 652/1322] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:50.806472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89610067-91a9-43d0-8d85-9362de27d79b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:50.806780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89610067-91a9-43d0-8d85-9362de27d79b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:50.807120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89610067-91a9-43d0-8d85-9362de27d79b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:50.808567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 671/1323] 10.222.0.55 () {54 vars in 964 bytes} [Sat Sep 24 19:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:50.814800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac9db-7ce7-488e-b3d6-39d7d248fd83 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:50.815078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac9db-7ce7-488e-b3d6-39d7d248fd83 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:50.815347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac9db-7ce7-488e-b3d6-39d7d248fd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:50.892211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-624ac9db-7ce7-488e-b3d6-39d7d248fd83 None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:50.893399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-624ac9db-7ce7-488e-b3d6-39d7d248fd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:50.968561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 653/1324] 10.222.0.55 () {58 vars in 1061 bytes} [Sat Sep 24 19:36:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:50.977861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89610067-91a9-43d0-8d85-9362de27d79b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:50.979521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-89610067-91a9-43d0-8d85-9362de27d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:51.000279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:51.092585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89610067-91a9-43d0-8d85-9362de27d79b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:51.172693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:51.173304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:51.173578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:51.173844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:51.175446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:51.251090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:51.289768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:51.291201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce525a-5674-4792-a705-84232c4b0f70 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:51.310194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 672/1325] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:36:53.785435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dac1ee-07ee-4ad5-b92d-62e798011a11 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:53.786306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dac1ee-07ee-4ad5-b92d-62e798011a11 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:53.786306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dac1ee-07ee-4ad5-b92d-62e798011a11 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:53.787987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 654/1326] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:53.793397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f9d6c-3775-4a30-85a3-bd6c9e68135e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:53.793834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f9d6c-3775-4a30-85a3-bd6c9e68135e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:53.794032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f9d6c-3775-4a30-85a3-bd6c9e68135e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:53.864025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7f4f9d6c-3775-4a30-85a3-bd6c9e68135e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:53.865161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7f4f9d6c-3775-4a30-85a3-bd6c9e68135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:53.965920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 673/1327] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:36:54.250254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dac1ee-07ee-4ad5-b92d-62e798011a11 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:54.251812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-53dac1ee-07ee-4ad5-b92d-62e798011a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:54.314882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:54.360789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dac1ee-07ee-4ad5-b92d-62e798011a11 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:54.439304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:54.439942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:54.440218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:54.440489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:54.442072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:54.564633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:54.567712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:54.570208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197739d-cf0e-43d1-9f4e-11beaac82138 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:54.590073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 655/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:36:56.861570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d13b-f55f-443e-a5ee-848a6ed7cca0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:56.862430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d13b-f55f-443e-a5ee-848a6ed7cca0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:56.862430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d13b-f55f-443e-a5ee-848a6ed7cca0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:56.864806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 674/1329] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:36:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:36:56.870312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a43a3-6ffd-4894-9452-032b95887e17 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:56.870950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a43a3-6ffd-4894-9452-032b95887e17 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:56.870950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a43a3-6ffd-4894-9452-032b95887e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:56.954437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-539a43a3-6ffd-4894-9452-032b95887e17 None None] MFA Rules not processed for user `475b2f85e68d43ac91521a15a826e5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:36:56.955717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-539a43a3-6ffd-4894-9452-032b95887e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:57.063112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 656/1330] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:36:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:36:57.351386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d13b-f55f-443e-a5ee-848a6ed7cca0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:36:57.352962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f380d13b-f55f-443e-a5ee-848a6ed7cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:57.385954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:57.464424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d13b-f55f-443e-a5ee-848a6ed7cca0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:36:57.542973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:36:57.543851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:36:57.544283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:36:57.544708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:36:57.546560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:36:57.638585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:36:57.674472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:36:57.675948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e43ba-e2c4-4a43-881f-1779962ca72f service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:36:57.695969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 675/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:36:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:00.238624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa428f-ecab-454b-88a2-599758f06f3e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:00.239221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa428f-ecab-454b-88a2-599758f06f3e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:00.239221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa428f-ecab-454b-88a2-599758f06f3e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:00.241512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 657/1332] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:00.247399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce0f1d-c6b9-43b6-abae-758896cc81c1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:00.247921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce0f1d-c6b9-43b6-abae-758896cc81c1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:00.248090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce0f1d-c6b9-43b6-abae-758896cc81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:00.315832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-88ce0f1d-c6b9-43b6-abae-758896cc81c1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:00.316954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-88ce0f1d-c6b9-43b6-abae-758896cc81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:00.412804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 676/1333] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:00.702410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa428f-ecab-454b-88a2-599758f06f3e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:00.704022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-64aa428f-ecab-454b-88a2-599758f06f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:00.782033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:00.817121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa428f-ecab-454b-88a2-599758f06f3e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:00.890309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:00.890991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:00.891305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:00.891656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:00.893391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:01.005080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:01.006449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69eeac5-ed4a-470d-b66a-155d3cdefa6d service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:01.016387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:01.025400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 658/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:03.265784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fbb70-a237-4acd-9f09-8523582b5f84 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:03.266302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fbb70-a237-4acd-9f09-8523582b5f84 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:03.266302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fbb70-a237-4acd-9f09-8523582b5f84 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:03.267596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 677/1335] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:03.272956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24a62-4efc-4abd-abf4-7edd53c45719 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:03.273401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24a62-4efc-4abd-abf4-7edd53c45719 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:03.273592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24a62-4efc-4abd-abf4-7edd53c45719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:03.339372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b9a24a62-4efc-4abd-abf4-7edd53c45719 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:03.340604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b9a24a62-4efc-4abd-abf4-7edd53c45719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:03.436209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 659/1336] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:03.724160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fbb70-a237-4acd-9f09-8523582b5f84 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:03.725872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-291fbb70-a237-4acd-9f09-8523582b5f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:03.766774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:03.829889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fbb70-a237-4acd-9f09-8523582b5f84 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:03.901126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:03.901725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:03.901995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:03.902261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:03.903859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:03.985145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:04.011120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:04.012591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9252dd-a5b3-4af2-a546-bd9c4b03277e service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:04.030151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 678/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:06.563949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8c3d7-315b-4899-bdc3-c797b6715806 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:06.564497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8c3d7-315b-4899-bdc3-c797b6715806 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:06.564497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8c3d7-315b-4899-bdc3-c797b6715806 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:06.565846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 660/1338] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:06.571350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2cc4c-eb24-438d-a0ec-08f7b34c3306 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:06.571800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2cc4c-eb24-438d-a0ec-08f7b34c3306 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:06.571958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2cc4c-eb24-438d-a0ec-08f7b34c3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:06.640641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6ed2cc4c-eb24-438d-a0ec-08f7b34c3306 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:06.641840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6ed2cc4c-eb24-438d-a0ec-08f7b34c3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:06.738973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 679/1339] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:07.023684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8c3d7-315b-4899-bdc3-c797b6715806 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:07.025266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-83f8c3d7-315b-4899-bdc3-c797b6715806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:07.103162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:07.130416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8c3d7-315b-4899-bdc3-c797b6715806 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:07.205099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:07.205703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:07.205980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:07.206246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:07.207873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:07.315634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:07.316994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a10e39-14f6-4e66-ad07-ab73c2ff9e86 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:07.335290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 661/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:11.888508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf524-b597-4775-a6fb-d1aa4dab1111 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:11.889064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf524-b597-4775-a6fb-d1aa4dab1111 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:11.889064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf524-b597-4775-a6fb-d1aa4dab1111 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:11.890488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 680/1341] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:11.895859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0691187-a9f4-456f-affc-d0a5f27500ed None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:11.896903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0691187-a9f4-456f-affc-d0a5f27500ed None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:11.897031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0691187-a9f4-456f-affc-d0a5f27500ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:11.903301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:11.968163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b0691187-a9f4-456f-affc-d0a5f27500ed None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:11.969354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b0691187-a9f4-456f-affc-d0a5f27500ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:12.070370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 662/1342] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:12.351960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf524-b597-4775-a6fb-d1aa4dab1111 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:12.353533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aafbf524-b597-4775-a6fb-d1aa4dab1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:12.406610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:12.464722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf524-b597-4775-a6fb-d1aa4dab1111 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:12.538535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:12.539137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:12.539445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:12.539710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:12.541291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:12.633967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:12.651576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:12.653015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fff61b-9171-46d6-b8e4-05053e0c2bbd service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:12.672276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 681/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:15.520658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5350ab1-976c-4386-8c0a-24e05e7bde7b None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:15.521756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5350ab1-976c-4386-8c0a-24e05e7bde7b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:15.521756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5350ab1-976c-4386-8c0a-24e05e7bde7b None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:15.523305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 663/1344] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:15.528471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e8be-998d-4ef6-a02c-14507cf328d4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:15.528875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e8be-998d-4ef6-a02c-14507cf328d4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:15.529016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e8be-998d-4ef6-a02c-14507cf328d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:15.594381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9280e8be-998d-4ef6-a02c-14507cf328d4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:15.595595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9280e8be-998d-4ef6-a02c-14507cf328d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:15.691880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 682/1345] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:15.985681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5350ab1-976c-4386-8c0a-24e05e7bde7b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:15.987196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f5350ab1-976c-4386-8c0a-24e05e7bde7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:16.080281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:16.101465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5350ab1-976c-4386-8c0a-24e05e7bde7b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:16.191033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:16.191661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:16.191940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:16.192201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:16.194560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:16.305852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:16.307512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f0d50-fdce-4509-a3f4-50f3888a800a service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:16.326682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 664/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:20.505922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645346a-052a-42aa-a5ec-cd06eba2d0a6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:20.506458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645346a-052a-42aa-a5ec-cd06eba2d0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:20.506655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645346a-052a-42aa-a5ec-cd06eba2d0a6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:20.508126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 683/1347] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:20.513329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50061f-057f-46e1-9bfb-f0469bde8f1c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:20.513744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50061f-057f-46e1-9bfb-f0469bde8f1c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:20.513856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50061f-057f-46e1-9bfb-f0469bde8f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:20.529465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:20.582475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9f50061f-057f-46e1-9bfb-f0469bde8f1c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:20.583733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9f50061f-057f-46e1-9bfb-f0469bde8f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:20.679597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 665/1348] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:20.961138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645346a-052a-42aa-a5ec-cd06eba2d0a6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:20.962642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1645346a-052a-42aa-a5ec-cd06eba2d0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:21.020647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:21.068891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645346a-052a-42aa-a5ec-cd06eba2d0a6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:21.141199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:21.141799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:21.142076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:21.142342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:21.143955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:21.253556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:21.258200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:21.259680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10ce35-320c-4719-96d4-0e008c1a486e service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:21.278214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 684/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:26.256182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a768c-fad2-4104-a4b2-53eb4b961580 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:26.256718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a768c-fad2-4104-a4b2-53eb4b961580 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:26.256718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a768c-fad2-4104-a4b2-53eb4b961580 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:26.258500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 666/1350] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:26.267634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9240-35c8-495b-83c5-bdac4b39e076 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:26.268130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9240-35c8-495b-83c5-bdac4b39e076 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:26.268340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9240-35c8-495b-83c5-bdac4b39e076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:26.369774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-799b9240-35c8-495b-83c5-bdac4b39e076 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:26.370972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-799b9240-35c8-495b-83c5-bdac4b39e076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:26.505859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 685/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:30.677465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=59 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:30.816876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11047bc4-5692-4349-87a5-80e2482dbecd None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:30.817401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11047bc4-5692-4349-87a5-80e2482dbecd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:30.818598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11047bc4-5692-4349-87a5-80e2482dbecd None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:30.820177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 667/1352] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:30.825473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295455-3e4f-4f14-9417-ff9eaf2fd2c9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:30.825942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295455-3e4f-4f14-9417-ff9eaf2fd2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:30.826041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295455-3e4f-4f14-9417-ff9eaf2fd2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:30.887361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:30.920515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-be295455-3e4f-4f14-9417-ff9eaf2fd2c9 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:30.922202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-be295455-3e4f-4f14-9417-ff9eaf2fd2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:31.043461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 686/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:37:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:32.366385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a61928-6da0-4073-9157-52bf0c356ebb None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:32.366920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a61928-6da0-4073-9157-52bf0c356ebb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:32.366920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a61928-6da0-4073-9157-52bf0c356ebb None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:32.368334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 668/1354] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:32.373455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6e0d-75eb-4c11-b81b-22405b78f574 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:32.373874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6e0d-75eb-4c11-b81b-22405b78f574 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:32.374044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6e0d-75eb-4c11-b81b-22405b78f574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:32.453741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-51ce6e0d-75eb-4c11-b81b-22405b78f574 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:32.454996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-51ce6e0d-75eb-4c11-b81b-22405b78f574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:32.498678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:32.566391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 687/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:37:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:36.291598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57dc81-2fdd-4b33-8f49-378664196c17 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:36.292518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57dc81-2fdd-4b33-8f49-378664196c17 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:36.292926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57dc81-2fdd-4b33-8f49-378664196c17 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:36.294579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 669/1356] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:36.300024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba75783-057a-447b-a6b2-a1d6c860e271 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:36.300482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba75783-057a-447b-a6b2-a1d6c860e271 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:36.300577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba75783-057a-447b-a6b2-a1d6c860e271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:36.376467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-aba75783-057a-447b-a6b2-a1d6c860e271 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:36.377717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aba75783-057a-447b-a6b2-a1d6c860e271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:36.480598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 688/1357] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:36.800160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ac627-f33c-418b-a258-7afe4bc6cd59 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:36.800597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ac627-f33c-418b-a258-7afe4bc6cd59 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:36.801014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ac627-f33c-418b-a258-7afe4bc6cd59 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:36.805971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 670/1358] 10.222.0.55 () {54 vars in 961 bytes} [Sat Sep 24 19:37:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:36.815464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a585d-15b5-4a4f-8c25-75f394851878 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:36.815852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a585d-15b5-4a4f-8c25-75f394851878 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:36.816315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a585d-15b5-4a4f-8c25-75f394851878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:36.846700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:36.926123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4e5a585d-15b5-4a4f-8c25-75f394851878 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:36.927638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4e5a585d-15b5-4a4f-8c25-75f394851878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:37.069266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 689/1359] 10.222.0.55 () {58 vars in 1058 bytes} [Sat Sep 24 19:37:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:37:37.070919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ac627-f33c-418b-a258-7afe4bc6cd59 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:37.074755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5e0ac627-f33c-418b-a258-7afe4bc6cd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:37.242863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ac627-f33c-418b-a258-7afe4bc6cd59 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:37.322863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:37.377390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:37.378170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:37.378554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:37.378921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:37.380949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:37.490292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:37.491953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce344390-57ac-4615-bb3a-a7b2043712c3 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:37.515161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 671/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:39.936008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11e62f-b220-4c9d-ac03-c50c7e0973ee None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:39.936608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11e62f-b220-4c9d-ac03-c50c7e0973ee None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:39.936608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11e62f-b220-4c9d-ac03-c50c7e0973ee None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:39.938028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 690/1361] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:37:39.943135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf6d76-b4ec-4529-862e-51d8a26d0336 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:39.943578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf6d76-b4ec-4529-862e-51d8a26d0336 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:39.943872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf6d76-b4ec-4529-862e-51d8a26d0336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:40.016363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-74cf6d76-b4ec-4529-862e-51d8a26d0336 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:40.017584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-74cf6d76-b4ec-4529-862e-51d8a26d0336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:40.062852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:40.131187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 672/1362] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:40.426174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11e62f-b220-4c9d-ac03-c50c7e0973ee None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:40.428014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2e11e62f-b220-4c9d-ac03-c50c7e0973ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:40.510725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:40.618183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11e62f-b220-4c9d-ac03-c50c7e0973ee None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:40.756400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:40.757103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:40.757379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:40.757647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:40.759255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:40.783045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:40.855925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:40.857541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c1f64-4e1b-4d6d-805c-c455d4040b91 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:40.876328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 691/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:43.432552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865a0bf-7fa5-4a0d-9edc-8d54bb03ade7 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:43.433135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865a0bf-7fa5-4a0d-9edc-8d54bb03ade7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:43.433135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865a0bf-7fa5-4a0d-9edc-8d54bb03ade7 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:43.434452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 673/1364] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:43.439903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d1e9-3b82-4c9a-83e4-ffce8ca1924a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:43.440356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d1e9-3b82-4c9a-83e4-ffce8ca1924a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:43.440461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d1e9-3b82-4c9a-83e4-ffce8ca1924a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:43.510285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-31c7d1e9-3b82-4c9a-83e4-ffce8ca1924a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:43.511594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-31c7d1e9-3b82-4c9a-83e4-ffce8ca1924a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:43.612302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:43.615356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 692/1365] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:37:43.910440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe06e5-2738-49af-9993-42fbba11488d None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:43.910730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe06e5-2738-49af-9993-42fbba11488d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:43.911005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe06e5-2738-49af-9993-42fbba11488d None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:43.912771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 674/1366] 10.222.0.55 () {54 vars in 961 bytes} [Sat Sep 24 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:43.918643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e93e07-1d90-4553-9278-40de6fec6b0e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:43.919030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e93e07-1d90-4553-9278-40de6fec6b0e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:43.919444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e93e07-1d90-4553-9278-40de6fec6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:44.000589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-90e93e07-1d90-4553-9278-40de6fec6b0e None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:44.001858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-90e93e07-1d90-4553-9278-40de6fec6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:44.136314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 693/1367] 10.222.0.55 () {58 vars in 1058 bytes} [Sat Sep 24 19:37:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:44.144184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe06e5-2738-49af-9993-42fbba11488d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:44.145727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9bfe06e5-2738-49af-9993-42fbba11488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:44.303115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe06e5-2738-49af-9993-42fbba11488d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:44.325211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:44.434458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:44.435121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:44.435380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:44.435637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:44.437258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:44.512467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1530ac1-201e-4e6c-9051-2fd299208447 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:44.513032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1530ac1-201e-4e6c-9051-2fd299208447 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:44.513032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1530ac1-201e-4e6c-9051-2fd299208447 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:44.514352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 694/1368] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:37:44.519259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64bec0-7c32-405a-89d6-c4a2a7864fa2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:44.519590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64bec0-7c32-405a-89d6-c4a2a7864fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:44.519873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64bec0-7c32-405a-89d6-c4a2a7864fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:44.535176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:44.537869 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45865cd-3f5e-4a31-b38b-3f7ef5c604ca service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:44.559815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 675/1369] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:37:44.605894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:44.617110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9f64bec0-7c32-405a-89d6-c4a2a7864fa2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:44.618394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9f64bec0-7c32-405a-89d6-c4a2a7864fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:44.745104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 695/1370] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:37:45.045298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51a42-9f23-4a39-bcbf-f6d70626add5 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:45.045580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51a42-9f23-4a39-bcbf-f6d70626add5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:45.045849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51a42-9f23-4a39-bcbf-f6d70626add5 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:45.047400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 676/1371] 10.222.0.55 () {54 vars in 963 bytes} [Sat Sep 24 19:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:45.052837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b55fc-26eb-4249-b217-7356a15519e0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:45.053460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b55fc-26eb-4249-b217-7356a15519e0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:45.053818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b55fc-26eb-4249-b217-7356a15519e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:45.143188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7f2b55fc-26eb-4249-b217-7356a15519e0 None None] MFA Rules not processed for user `b93ea821819d462097b1e76ce1ffae3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:45.144757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7f2b55fc-26eb-4249-b217-7356a15519e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:45.228767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:45.238328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 696/1372] 10.222.0.55 () {58 vars in 1060 bytes} [Sat Sep 24 19:37:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:45.246942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51a42-9f23-4a39-bcbf-f6d70626add5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:45.248490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-72b51a42-9f23-4a39-bcbf-f6d70626add5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:45.278758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:45.376824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51a42-9f23-4a39-bcbf-f6d70626add5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:45.462865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:45.463405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:45.463847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:45.464163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:45.465653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:45.559188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:45.596480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:45.597957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e05a0c-8dd2-4a98-b1cc-6a50c952b2a9 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:45.617184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 677/1373] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:46.852968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53752ee0-fd77-409d-94b1-f189abb2cc3b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:46.853565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53752ee0-fd77-409d-94b1-f189abb2cc3b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:46.853565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53752ee0-fd77-409d-94b1-f189abb2cc3b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:46.854845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 697/1374] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:46.860218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae6d0a-c0d8-4883-a453-733dc515cce3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:46.860611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae6d0a-c0d8-4883-a453-733dc515cce3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:46.860760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae6d0a-c0d8-4883-a453-733dc515cce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:46.932273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-82ae6d0a-c0d8-4883-a453-733dc515cce3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:46.933492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-82ae6d0a-c0d8-4883-a453-733dc515cce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:47.032219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 678/1375] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:47.326580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53752ee0-fd77-409d-94b1-f189abb2cc3b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:47.328177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-53752ee0-fd77-409d-94b1-f189abb2cc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:47.482590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53752ee0-fd77-409d-94b1-f189abb2cc3b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:47.558822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:47.609017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:47.609622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:47.609913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:47.610166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:47.612195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:47.714577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:47.717175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cf348-8c62-455c-a6d5-350543c445be service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:47.743138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 698/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:50.007994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f32a04-2594-445a-be66-501a1ceb4fb0 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:50.008531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f32a04-2594-445a-be66-501a1ceb4fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:50.008799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f32a04-2594-445a-be66-501a1ceb4fb0 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:50.011216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 679/1377] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:50.016857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e80c8-8da9-469b-91ae-fb2574a35dc7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:50.017417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e80c8-8da9-469b-91ae-fb2574a35dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:50.017417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e80c8-8da9-469b-91ae-fb2574a35dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:50.096367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9f4e80c8-8da9-469b-91ae-fb2574a35dc7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:50.097968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9f4e80c8-8da9-469b-91ae-fb2574a35dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:50.129645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:50.211478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 699/1378] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:37:50.506291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f32a04-2594-445a-be66-501a1ceb4fb0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:50.508128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-92f32a04-2594-445a-be66-501a1ceb4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:50.547640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:50.667925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f32a04-2594-445a-be66-501a1ceb4fb0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:50.808349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:50.815124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:50.815874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:50.816249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:50.816636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:50.819069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:50.920491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:50.922292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ed496-a1b3-4c8f-8d59-fc6ab9432c3b service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:50.943594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 680/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:53.221633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56aade-2d14-4013-b04b-94ae4c96760c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:53.222136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56aade-2d14-4013-b04b-94ae4c96760c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:53.222136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56aade-2d14-4013-b04b-94ae4c96760c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:53.223476 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 700/1380] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:53.229704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93558fd-1880-4e81-bf8d-7a20a28cc4c4 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:53.230185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93558fd-1880-4e81-bf8d-7a20a28cc4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:53.230510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93558fd-1880-4e81-bf8d-7a20a28cc4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:53.305967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-d93558fd-1880-4e81-bf8d-7a20a28cc4c4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:53.307514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d93558fd-1880-4e81-bf8d-7a20a28cc4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:53.382381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:53.424816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 681/1381] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:53.748857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56aade-2d14-4013-b04b-94ae4c96760c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:53.750361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fd56aade-2d14-4013-b04b-94ae4c96760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:53.918228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56aade-2d14-4013-b04b-94ae4c96760c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:53.932788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:54.045344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:54.045934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:54.046238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:54.046538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:54.048376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:54.150684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:54.152318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120b67-85c8-4853-9e49-8c6154cad763 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:54.171043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 701/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:37:56.991758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c083-9623-4513-8053-83dbe7a3e0ae None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:56.992357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c083-9623-4513-8053-83dbe7a3e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:56.992464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c083-9623-4513-8053-83dbe7a3e0ae None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:56.994190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 682/1383] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:37:57.000548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e8021-67a0-408f-a04e-5868c669b3ea None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:57.001041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e8021-67a0-408f-a04e-5868c669b3ea None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:57.001150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e8021-67a0-408f-a04e-5868c669b3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:57.025268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:57.077896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-242e8021-67a0-408f-a04e-5868c669b3ea None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:37:57.079218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-242e8021-67a0-408f-a04e-5868c669b3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:57.195285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 702/1384] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:37:57.502839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95c083-9623-4513-8053-83dbe7a3e0ae None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:37:57.504791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bf95c083-9623-4513-8053-83dbe7a3e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:57.666639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95c083-9623-4513-8053-83dbe7a3e0ae None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:37:57.726150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:37:57.793598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:37:57.794252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:37:57.794546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:37:57.794831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:37:57.796559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:37:57.895940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:37:57.897617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a124bc-86f5-4601-85e1-e0226b824179 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:37:57.918695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 683/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:00.342104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40860673-ec09-4e0a-b515-f82e0f46df11 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:00.342674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40860673-ec09-4e0a-b515-f82e0f46df11 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:00.342674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40860673-ec09-4e0a-b515-f82e0f46df11 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:00.343910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 703/1386] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:00.349513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c782a-c202-4fbe-b293-a1028fd6a00f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:00.349975 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c782a-c202-4fbe-b293-a1028fd6a00f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:00.350207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c782a-c202-4fbe-b293-a1028fd6a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:00.418652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-032c782a-c202-4fbe-b293-a1028fd6a00f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:00.419874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-032c782a-c202-4fbe-b293-a1028fd6a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:00.428827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:00.519059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 684/1387] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:00.809233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40860673-ec09-4e0a-b515-f82e0f46df11 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:00.810869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-40860673-ec09-4e0a-b515-f82e0f46df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:00.906657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:00.963729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40860673-ec09-4e0a-b515-f82e0f46df11 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:01.090213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:01.090831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:01.091117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:01.091535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:01.093126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:01.159667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:01.193769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:01.194069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278de639-dedc-4545-a1b2-257e80f76966 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:01.194352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278de639-dedc-4545-a1b2-257e80f76966 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:01.194621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278de639-dedc-4545-a1b2-257e80f76966 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:01.195376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbfc55-5499-40c3-92c7-901f13687ff6 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:01.196021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 685/1388] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:01.201335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa9140-6e04-4a7e-953d-f0941dccfa63 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:01.201611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa9140-6e04-4a7e-953d-f0941dccfa63 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:01.201878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa9140-6e04-4a7e-953d-f0941dccfa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:01.214874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 704/1389] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:01.280493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7daa9140-6e04-4a7e-953d-f0941dccfa63 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:01.281882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7daa9140-6e04-4a7e-953d-f0941dccfa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:01.386784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:01.400538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 686/1390] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:38:01.704503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd915b-2774-4417-892a-ac343e14b6db None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:01.704985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd915b-2774-4417-892a-ac343e14b6db None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:01.705179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd915b-2774-4417-892a-ac343e14b6db None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:01.707048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 705/1391] 10.222.0.55 () {54 vars in 962 bytes} [Sat Sep 24 19:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:01.713305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64ad38-c84f-4717-b5e1-9a0d5009581d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:01.713659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64ad38-c84f-4717-b5e1-9a0d5009581d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:01.713997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64ad38-c84f-4717-b5e1-9a0d5009581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:01.794719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0f64ad38-c84f-4717-b5e1-9a0d5009581d None None] MFA Rules not processed for user `6e9ce9098e884f4e9119aeccdbafd4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:01.795947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0f64ad38-c84f-4717-b5e1-9a0d5009581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:01.874119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 687/1392] 10.222.0.55 () {58 vars in 1059 bytes} [Sat Sep 24 19:38:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:01.884088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd915b-2774-4417-892a-ac343e14b6db None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:01.885592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-07bd915b-2774-4417-892a-ac343e14b6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:01.992156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd915b-2774-4417-892a-ac343e14b6db None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:02.080701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:02.080942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:02.081109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:02.081370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:02.083027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:02.091924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:02.203698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:02.205144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80d5d7-4fcd-427e-9575-60df79a8fcd7 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:02.223239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 706/1393] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:02.983690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93baa36-a7c5-43fd-bda2-abad3f395028 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:02.984232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93baa36-a7c5-43fd-bda2-abad3f395028 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:02.984554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93baa36-a7c5-43fd-bda2-abad3f395028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:03.056765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e93baa36-a7c5-43fd-bda2-abad3f395028 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:03.058333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e93baa36-a7c5-43fd-bda2-abad3f395028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:03.099028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:03.156953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 688/1394] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:38:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:03.201107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1734a0b-f166-4053-855c-47d139b1279e None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:03.201615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1734a0b-f166-4053-855c-47d139b1279e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:03.201718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1734a0b-f166-4053-855c-47d139b1279e None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:03.203035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 707/1395] 10.222.0.55 () {54 vars in 943 bytes} [Sat Sep 24 19:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:03.210458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3598cb-7924-4c8e-8e79-9e934373c54b None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:03.210897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3598cb-7924-4c8e-8e79-9e934373c54b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:03.211459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3598cb-7924-4c8e-8e79-9e934373c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:03.294916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-af3598cb-7924-4c8e-8e79-9e934373c54b None None] MFA Rules not processed for user `c72b7d859e954054848d405afa0b466f`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:03.296068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-af3598cb-7924-4c8e-8e79-9e934373c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:03.420127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 689/1396] 10.222.0.55 () {58 vars in 1040 bytes} [Sat Sep 24 19:38:03 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:03.430510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1734a0b-f166-4053-855c-47d139b1279e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:03.432337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d1734a0b-f166-4053-855c-47d139b1279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:03.469222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d954f-10d6-441f-9a98-071e1f4bec79 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:03.469508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d954f-10d6-441f-9a98-071e1f4bec79 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:03.469793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d954f-10d6-441f-9a98-071e1f4bec79 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:03.471159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 690/1397] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:38:03.476930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:03.477210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:03.477481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:03.487030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:03.548617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:03.551107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:03.553335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:03.578206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1734a0b-f166-4053-855c-47d139b1279e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:03.655889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 691/1398] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:38:03.686804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:03.687402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:03.687704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:03.687969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:03.689572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:03.776860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:03.778485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7ec8f-e1a8-4e00-810d-c16154515c5d service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:03.786546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 708/1399] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:38:03.968523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:03.975468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:04.076837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:04.117056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d061c1-012c-443e-9776-91fb063ccc19 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:04.247286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:04.247997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:04.248290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:04.248557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:04.250151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:04.341859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:04.354848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:04.356489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10db7f-d971-4cbd-b121-2f06cbd95af6 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:04.377687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 692/1400] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:06.718826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014aba1-f891-4c05-b126-4cab2ecd69c2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:06.719362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014aba1-f891-4c05-b126-4cab2ecd69c2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:06.719362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014aba1-f891-4c05-b126-4cab2ecd69c2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:06.720874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 709/1401] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:06.726283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bf2ff-59c8-4b92-9274-027528a634b2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:06.726748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bf2ff-59c8-4b92-9274-027528a634b2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:06.726852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bf2ff-59c8-4b92-9274-027528a634b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:06.798246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ba5bf2ff-59c8-4b92-9274-027528a634b2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:06.799393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ba5bf2ff-59c8-4b92-9274-027528a634b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:06.906349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 693/1402] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:07.200918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014aba1-f891-4c05-b126-4cab2ecd69c2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:07.202477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9014aba1-f891-4c05-b126-4cab2ecd69c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:07.219658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:07.372133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014aba1-f891-4c05-b126-4cab2ecd69c2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:07.471289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:07.516902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:07.517811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:07.518238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:07.518659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:07.520906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:07.627036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:07.628567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ed4a2-3248-4264-9cbb-12ad477281dc service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:07.648919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 710/1403] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:09.923115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9904dcb-e200-4afd-a5d9-80f7e10892c6 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:09.923717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9904dcb-e200-4afd-a5d9-80f7e10892c6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:09.923717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9904dcb-e200-4afd-a5d9-80f7e10892c6 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:09.925073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 694/1404] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:38:09.931591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37184608-0587-4033-8557-7207cb2a71f4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:09.932088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37184608-0587-4033-8557-7207cb2a71f4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:09.932424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37184608-0587-4033-8557-7207cb2a71f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:10.014658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-37184608-0587-4033-8557-7207cb2a71f4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:10.015937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-37184608-0587-4033-8557-7207cb2a71f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:10.080113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:10.142637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 711/1405] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:10.462447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9904dcb-e200-4afd-a5d9-80f7e10892c6 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:10.464012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c9904dcb-e200-4afd-a5d9-80f7e10892c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:10.550885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:10.663156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9904dcb-e200-4afd-a5d9-80f7e10892c6 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:10.815797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:10.816531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:10.816823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:10.817150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:10.818838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:10.844290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:10.922579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:10.924384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca191-2184-40c2-9a65-4840c832abf2 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:10.945358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 695/1406] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:13.203862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30151ba1-eaa5-4337-83aa-d6565d4b7e22 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:13.204400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30151ba1-eaa5-4337-83aa-d6565d4b7e22 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:13.204400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30151ba1-eaa5-4337-83aa-d6565d4b7e22 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:13.205861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 712/1407] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:13.211541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faabb6-17dc-4ee7-878d-7e6a5367e4a7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:13.212308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faabb6-17dc-4ee7-878d-7e6a5367e4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:13.212744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faabb6-17dc-4ee7-878d-7e6a5367e4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:13.287745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-50faabb6-17dc-4ee7-878d-7e6a5367e4a7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:13.289682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-50faabb6-17dc-4ee7-878d-7e6a5367e4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:13.395910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:13.399685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 696/1408] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:13.692172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30151ba1-eaa5-4337-83aa-d6565d4b7e22 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:13.694271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-30151ba1-eaa5-4337-83aa-d6565d4b7e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:13.864737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30151ba1-eaa5-4337-83aa-d6565d4b7e22 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:13.902356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:14.000749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:14.001402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:14.001739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:14.002085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:14.003877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:14.111373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:14.113122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598d19-5810-42c1-aee2-fdf5416f1084 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:14.133215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 713/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:23.408910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:24.130716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:27.463244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec1711-5df3-4982-82c5-5a354021c73b None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:27.463867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec1711-5df3-4982-82c5-5a354021c73b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:27.463867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec1711-5df3-4982-82c5-5a354021c73b None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:27.465217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 697/1410] 10.222.0.55 () {54 vars in 918 bytes} [Sat Sep 24 19:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:27.470496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ca041-e7a7-44ca-9cb4-a60dce8ede18 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:27.470937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ca041-e7a7-44ca-9cb4-a60dce8ede18 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:27.471066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ca041-e7a7-44ca-9cb4-a60dce8ede18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:27.548298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-013ca041-e7a7-44ca-9cb4-a60dce8ede18 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:27.549521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-013ca041-e7a7-44ca-9cb4-a60dce8ede18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:27.659553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 714/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Sat Sep 24 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:27.673997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32b2d2-d587-4dfb-b8d8-2df8cd852de8 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:27.674283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32b2d2-d587-4dfb-b8d8-2df8cd852de8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:27.674555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32b2d2-d587-4dfb-b8d8-2df8cd852de8 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:27.675950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 698/1412] 10.222.0.55 () {54 vars in 918 bytes} [Sat Sep 24 19:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:27.681534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfef05-7dad-45b9-981f-00ade842b797 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:27.681826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfef05-7dad-45b9-981f-00ade842b797 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:27.682101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfef05-7dad-45b9-981f-00ade842b797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:27.757379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:27.760301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8edfef05-7dad-45b9-981f-00ade842b797 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:27.762100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8edfef05-7dad-45b9-981f-00ade842b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:27.878907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 715/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Sat Sep 24 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:30.506204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9fc98-3ed5-46b9-93c8-cfb5734988ca None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:30.506798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9fc98-3ed5-46b9-93c8-cfb5734988ca None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:30.507081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9fc98-3ed5-46b9-93c8-cfb5734988ca None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:30.508558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:30.513994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443fd20-2f1d-4c9d-afd5-c768a3f3db3a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:30.514461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443fd20-2f1d-4c9d-afd5-c768a3f3db3a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:30.514557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443fd20-2f1d-4c9d-afd5-c768a3f3db3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:30.583395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6443fd20-2f1d-4c9d-afd5-c768a3f3db3a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:30.584636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6443fd20-2f1d-4c9d-afd5-c768a3f3db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:30.640903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:30.683811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:30.976611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9fc98-3ed5-46b9-93c8-cfb5734988ca None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:30.978213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b7d9fc98-3ed5-46b9-93c8-cfb5734988ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:31.125720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9fc98-3ed5-46b9-93c8-cfb5734988ca None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:31.201666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:31.242250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:31.242865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:31.243145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:31.243414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:31.245041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:31.335862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:31.337507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08826a9-c10a-479f-bfc9-a0e18cbab4f7 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:31.355483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 700/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:32.497214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b7ed-721c-4648-8c76-9b5bc5e22c05 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:32.497806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b7ed-721c-4648-8c76-9b5bc5e22c05 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:32.497806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b7ed-721c-4648-8c76-9b5bc5e22c05 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:32.499100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:32.505842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb5453-b07b-465c-b6c1-b84043aea0e3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:32.506331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb5453-b07b-465c-b6c1-b84043aea0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:32.506618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb5453-b07b-465c-b6c1-b84043aea0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:32.583552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-60cb5453-b07b-465c-b6c1-b84043aea0e3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:32.584758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-60cb5453-b07b-465c-b6c1-b84043aea0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:32.611212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:32.682058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:38:32.830417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4b7ed-721c-4648-8c76-9b5bc5e22c05 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:32.831855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d3a4b7ed-721c-4648-8c76-9b5bc5e22c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:32.984291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4b7ed-721c-4648-8c76-9b5bc5e22c05 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:33.025634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:33.100716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:33.101241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:33.101515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:33.101782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:33.103347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:33.193113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:33.194691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a826c-2a60-439c-a052-88c7676aace8 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:33.212563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 718/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:34.438346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86449e0-e254-476e-8ebe-2acb112f1970 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:34.438894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86449e0-e254-476e-8ebe-2acb112f1970 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:34.438894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86449e0-e254-476e-8ebe-2acb112f1970 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:34.440276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:34.448335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fdf6a-d5d9-4541-8659-e89c8e2cbfda None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:34.448863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fdf6a-d5d9-4541-8659-e89c8e2cbfda None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:34.448978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fdf6a-d5d9-4541-8659-e89c8e2cbfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:34.502679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:34.526303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b16fdf6a-d5d9-4541-8659-e89c8e2cbfda None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:34.527566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b16fdf6a-d5d9-4541-8659-e89c8e2cbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:34.626704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:34.788518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86449e0-e254-476e-8ebe-2acb112f1970 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:34.789938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b86449e0-e254-476e-8ebe-2acb112f1970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:34.937824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86449e0-e254-476e-8ebe-2acb112f1970 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:34.949871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:35.054582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:35.055174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:35.055390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:35.055952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:35.057579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:35.151556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:35.153169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0b4ba-37f2-4b57-80ad-273a31c910e0 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:35.173558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 703/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:36.281116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bc5a0-9888-4aa6-8418-e26ff1cba136 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:36.281660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bc5a0-9888-4aa6-8418-e26ff1cba136 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:36.281660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bc5a0-9888-4aa6-8418-e26ff1cba136 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:36.283036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:38:36.289594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f7f0d-3a72-4bac-b1ea-922647269fe7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:36.290078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f7f0d-3a72-4bac-b1ea-922647269fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:36.290227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f7f0d-3a72-4bac-b1ea-922647269fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:36.310102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:36.367525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5e5f7f0d-3a72-4bac-b1ea-922647269fe7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:36.368665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5e5f7f0d-3a72-4bac-b1ea-922647269fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:36.465742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:36.618670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bc5a0-9888-4aa6-8418-e26ff1cba136 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:36.620147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5f1bc5a0-9888-4aa6-8418-e26ff1cba136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:36.731941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:36.769422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bc5a0-9888-4aa6-8418-e26ff1cba136 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:36.882026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:36.882540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:36.882816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:36.883088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:36.884713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:36.961583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:36.975700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:36.977346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edeb0a-ac02-49a7-884e-f8f26ad6ccd7 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:36.995523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 721/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:38.123043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8269-93da-4cb3-93e5-9b6052613eaa None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:38.123656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8269-93da-4cb3-93e5-9b6052613eaa None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:38.123656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8269-93da-4cb3-93e5-9b6052613eaa None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:38.125028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:38.132042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be9524-cf89-476d-811b-91bd4280c3e0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:38.132494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be9524-cf89-476d-811b-91bd4280c3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:38.132609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be9524-cf89-476d-811b-91bd4280c3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:38.208839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-55be9524-cf89-476d-811b-91bd4280c3e0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:38.210112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-55be9524-cf89-476d-811b-91bd4280c3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:38.307832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:38.455601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c8269-93da-4cb3-93e5-9b6052613eaa None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:38.457163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3d6c8269-93da-4cb3-93e5-9b6052613eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:38.545895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:38.611603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c8269-93da-4cb3-93e5-9b6052613eaa None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:38.725507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:38.726122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:38.726401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:38.726701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:38.728337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:38.780501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:38.830013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:38.832355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951c356-5792-4bdd-b5dd-4eb728195771 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:38.851071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 706/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:39.902760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8dbbb-6505-4699-addf-f49cec804013 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:39.903306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8dbbb-6505-4699-addf-f49cec804013 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:39.903306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8dbbb-6505-4699-addf-f49cec804013 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:39.904674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:39.911981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f9d6-819f-49ee-8e8d-a9a5952e1dd1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:39.912448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f9d6-819f-49ee-8e8d-a9a5952e1dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:39.912583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f9d6-819f-49ee-8e8d-a9a5952e1dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:39.990447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-3107f9d6-819f-49ee-8e8d-a9a5952e1dd1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:39.991712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3107f9d6-819f-49ee-8e8d-a9a5952e1dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:40.090768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:40.242614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8dbbb-6505-4699-addf-f49cec804013 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:40.244187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a0c8dbbb-6505-4699-addf-f49cec804013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:40.296536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:40.394567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8dbbb-6505-4699-addf-f49cec804013 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:40.507535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:40.508142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:40.508424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:40.508692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:40.510268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:40.517054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:40.601618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:40.603208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc2990-4d57-4795-b898-391b3b8334ef service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:40.621976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 724/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:41.687769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a863a-b1d8-4480-874d-b40243c06921 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:41.688353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a863a-b1d8-4480-874d-b40243c06921 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:41.688353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a863a-b1d8-4480-874d-b40243c06921 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:41.689695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:41.696394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373eb1a-863a-4543-80d9-1819f0c65919 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:41.696853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373eb1a-863a-4543-80d9-1819f0c65919 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:41.696952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373eb1a-863a-4543-80d9-1819f0c65919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:41.774892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2373eb1a-863a-4543-80d9-1819f0c65919 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:41.776120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2373eb1a-863a-4543-80d9-1819f0c65919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:41.844177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:41.873969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:42.025591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a863a-b1d8-4480-874d-b40243c06921 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:42.027110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a94a863a-b1d8-4480-874d-b40243c06921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:42.046680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:42.184547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a863a-b1d8-4480-874d-b40243c06921 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:42.271674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:42.298067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:42.298600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:42.298875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:42.299139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:42.301333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:42.390664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:42.392295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b9d3f-1311-46c5-bcee-0454c9fc29c3 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:42.410956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 709/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:43.458919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ff08e-4b14-45cc-99f5-985f91bfc614 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:43.459490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ff08e-4b14-45cc-99f5-985f91bfc614 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:43.459490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ff08e-4b14-45cc-99f5-985f91bfc614 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:43.460805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:43.466936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb99f-b047-41dc-b0bc-30585dd7e45f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:43.467399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb99f-b047-41dc-b0bc-30585dd7e45f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:43.467527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb99f-b047-41dc-b0bc-30585dd7e45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:43.544100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-249cb99f-b047-41dc-b0bc-30585dd7e45f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:43.545305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-249cb99f-b047-41dc-b0bc-30585dd7e45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:43.588764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:43.650688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:43.801419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ff08e-4b14-45cc-99f5-985f91bfc614 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:43.803000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3e7ff08e-4b14-45cc-99f5-985f91bfc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:43.952619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ff08e-4b14-45cc-99f5-985f91bfc614 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:44.007957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:44.068614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:44.069237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:44.069515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:44.069784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:44.071408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:44.162657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:44.164274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dced38-a46b-4c3b-9e62-9371c4f2b4c5 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:44.182641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 727/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:45.235261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff672b-3189-47d1-baf9-d38fd8f97419 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:45.235830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff672b-3189-47d1-baf9-d38fd8f97419 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:45.235830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff672b-3189-47d1-baf9-d38fd8f97419 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:45.237826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:45.244263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43cd6e-e8c1-424d-8af2-041b20e8056f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:45.244702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43cd6e-e8c1-424d-8af2-041b20e8056f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:45.244828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43cd6e-e8c1-424d-8af2-041b20e8056f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:45.321784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0c43cd6e-e8c1-424d-8af2-041b20e8056f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:45.322945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0c43cd6e-e8c1-424d-8af2-041b20e8056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:45.325014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:45.421717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:45.575980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff672b-3189-47d1-baf9-d38fd8f97419 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:45.577477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f6ff672b-3189-47d1-baf9-d38fd8f97419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:45.724362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff672b-3189-47d1-baf9-d38fd8f97419 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:45.751669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:45.843586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:45.844046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:45.844319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:45.844583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:45.846157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:45.935414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:45.937051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeee59-779c-45e5-be09-588188e25752 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:45.955159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 712/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:48.025894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253c9c2-72f4-46bb-8fa5-5b73d0ee1e2f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:48.026414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253c9c2-72f4-46bb-8fa5-5b73d0ee1e2f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:48.026414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253c9c2-72f4-46bb-8fa5-5b73d0ee1e2f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:48.027853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:48.033352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04efa28-2bf0-4025-9ad0-577d7662ef3a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:48.033883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04efa28-2bf0-4025-9ad0-577d7662ef3a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:48.034042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04efa28-2bf0-4025-9ad0-577d7662ef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:48.076410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:48.103186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f04efa28-2bf0-4025-9ad0-577d7662ef3a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:48.104692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f04efa28-2bf0-4025-9ad0-577d7662ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:48.201985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:48.493230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253c9c2-72f4-46bb-8fa5-5b73d0ee1e2f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:48.494754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4253c9c2-72f4-46bb-8fa5-5b73d0ee1e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:48.597377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253c9c2-72f4-46bb-8fa5-5b73d0ee1e2f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:48.630216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:48.670829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:48.671438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:48.671727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:48.671992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:48.673566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:48.786233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:48.787729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eac64-e534-401d-8e2e-ecb86d043be9 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:48.806698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 730/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:51.174638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e996c5-a54e-404f-b422-d7739556681c None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:51.175210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e996c5-a54e-404f-b422-d7739556681c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:51.175210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e996c5-a54e-404f-b422-d7739556681c None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:51.176560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:51.182008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba51e-e3f0-4e54-8995-1f0f70e21ca3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:51.182449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba51e-e3f0-4e54-8995-1f0f70e21ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:51.182547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba51e-e3f0-4e54-8995-1f0f70e21ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:51.248797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-890ba51e-e3f0-4e54-8995-1f0f70e21ca3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:51.250011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-890ba51e-e3f0-4e54-8995-1f0f70e21ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:51.263640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:51.350092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:51.635417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e996c5-a54e-404f-b422-d7739556681c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:51.636972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d0e996c5-a54e-404f-b422-d7739556681c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:51.742027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e996c5-a54e-404f-b422-d7739556681c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:51.755953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:51.819459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:51.820064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:51.820339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:51.820604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:51.822186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:51.930727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:51.932205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322567d2-2c35-416d-b552-9020e1d7bc45 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:51.951533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 715/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:54.514077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2469d34-2432-4491-9156-fb775655629c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:54.514652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2469d34-2432-4491-9156-fb775655629c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:54.514652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2469d34-2432-4491-9156-fb775655629c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:54.515978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:54.521690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df1a57-a9cd-49af-a93e-2ca77e92c536 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:54.522195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df1a57-a9cd-49af-a93e-2ca77e92c536 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:54.522295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df1a57-a9cd-49af-a93e-2ca77e92c536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:54.570282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:54.589726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-96df1a57-a9cd-49af-a93e-2ca77e92c536 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:54.590954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-96df1a57-a9cd-49af-a93e-2ca77e92c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:54.688881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:54.979976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2469d34-2432-4491-9156-fb775655629c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:54.981520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e2469d34-2432-4491-9156-fb775655629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:55.086915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2469d34-2432-4491-9156-fb775655629c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:55.126124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:55.162131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:55.162760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:55.163031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:55.163297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:55.164953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:55.275070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:55.276567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fedbb-d6b4-4d12-9951-799e26da604c service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:55.294189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 733/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:38:57.599090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7410b7-e00f-4c6e-adb4-2dd1bc6418f4 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:57.599697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7410b7-e00f-4c6e-adb4-2dd1bc6418f4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:57.599697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7410b7-e00f-4c6e-adb4-2dd1bc6418f4 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:57.601006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:38:57.606293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf200a-ca3b-4193-b2ce-fd8a3584e6db None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:57.606770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf200a-ca3b-4193-b2ce-fd8a3584e6db None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:57.606870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf200a-ca3b-4193-b2ce-fd8a3584e6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:57.672322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ddaf200a-ca3b-4193-b2ce-fd8a3584e6db None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:38:57.674052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ddaf200a-ca3b-4193-b2ce-fd8a3584e6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:57.695406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:57.775639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:38:58.066623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7410b7-e00f-4c6e-adb4-2dd1bc6418f4 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:38:58.068279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1c7410b7-e00f-4c6e-adb4-2dd1bc6418f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:58.176593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7410b7-e00f-4c6e-adb4-2dd1bc6418f4 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:38:58.201328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:38:58.261503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:38:58.262112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:38:58.262392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:38:58.262663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:38:58.264291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:38:58.376713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:38:58.378422 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c472-fa56-4848-91b1-9215bb51f999 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:38:58.396967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 718/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:00.695539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3de4-74f0-4111-828a-8155ce0b71a2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:00.696166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3de4-74f0-4111-828a-8155ce0b71a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:00.696166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3de4-74f0-4111-828a-8155ce0b71a2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:00.697401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:00.702709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2303a-d276-4a73-91e2-f9c502816bd5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:00.703160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2303a-d276-4a73-91e2-f9c502816bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:00.703310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2303a-d276-4a73-91e2-f9c502816bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:00.904566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:00.908277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-dcb2303a-d276-4a73-91e2-f9c502816bd5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:00.909884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dcb2303a-d276-4a73-91e2-f9c502816bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:01.008425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:01.292027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3de4-74f0-4111-828a-8155ce0b71a2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:01.293536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-304f3de4-74f0-4111-828a-8155ce0b71a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:01.397787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3de4-74f0-4111-828a-8155ce0b71a2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:01.448173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:01.474182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:01.474944 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:01.475307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:01.475703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:01.477396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:01.589661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:01.590984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248a7db-4582-49e3-b2b5-b7f30d018061 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:01.609751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 736/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:03.840284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37692b-e568-452e-8516-291b1f77ba23 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:03.840857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37692b-e568-452e-8516-291b1f77ba23 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:03.840857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37692b-e568-452e-8516-291b1f77ba23 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:03.842154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:39:03.847824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43650ab6-3578-4c4d-adf3-1c899da08afa None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:03.848282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43650ab6-3578-4c4d-adf3-1c899da08afa None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:03.848545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43650ab6-3578-4c4d-adf3-1c899da08afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:03.917335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-43650ab6-3578-4c4d-adf3-1c899da08afa None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:03.918553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-43650ab6-3578-4c4d-adf3-1c899da08afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:03.954168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:04.025219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:04.316993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37692b-e568-452e-8516-291b1f77ba23 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:04.318442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3d37692b-e568-452e-8516-291b1f77ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:04.424524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37692b-e568-452e-8516-291b1f77ba23 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:04.454139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:04.500726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:04.502076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:04.502602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:04.502961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:04.505090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:04.641941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:04.643310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09fbf6-fd8d-4ddb-8017-ea16eba646a9 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:04.662021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 721/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:06.958857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b88667-d4fa-4bd3-8562-b08d94de6e7a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:06.959446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b88667-d4fa-4bd3-8562-b08d94de6e7a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:06.959446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b88667-d4fa-4bd3-8562-b08d94de6e7a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:06.960703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:06.966614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2c58e-91fd-4bff-b8af-1f23c265cf7c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:06.967040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2c58e-91fd-4bff-b8af-1f23c265cf7c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:06.967228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2c58e-91fd-4bff-b8af-1f23c265cf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:07.033279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-68a2c58e-91fd-4bff-b8af-1f23c265cf7c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:07.034487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-68a2c58e-91fd-4bff-b8af-1f23c265cf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:07.043040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:07.134912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:07.422187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b88667-d4fa-4bd3-8562-b08d94de6e7a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:07.424318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-48b88667-d4fa-4bd3-8562-b08d94de6e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:07.528701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b88667-d4fa-4bd3-8562-b08d94de6e7a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:07.585473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:07.604768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:07.605366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:07.605639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:07.605904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:07.607507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:07.720748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:07.722091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76792c7a-6fe8-461c-99fe-6a6eff736de2 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:07.740635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 739/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:10.077945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef517070-8b0e-4d80-82d7-a54d654ba1c7 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:10.078510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef517070-8b0e-4d80-82d7-a54d654ba1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:10.078510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef517070-8b0e-4d80-82d7-a54d654ba1c7 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:10.079912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:39:10.085273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d532b-5faa-4499-a3b6-15479a45627d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:10.085698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d532b-5faa-4499-a3b6-15479a45627d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:10.085831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d532b-5faa-4499-a3b6-15479a45627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:10.271088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6b9d532b-5faa-4499-a3b6-15479a45627d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:10.272337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6b9d532b-5faa-4499-a3b6-15479a45627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:10.311630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:10.371271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:10.659207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef517070-8b0e-4d80-82d7-a54d654ba1c7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:10.660850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ef517070-8b0e-4d80-82d7-a54d654ba1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:10.773251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef517070-8b0e-4d80-82d7-a54d654ba1c7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:10.817904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:10.864003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:10.864639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:10.864940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:10.865361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:10.866994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:11.008623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:11.010105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6016d-2d81-419f-99e2-1c677d138bc6 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:11.030605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 724/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:13.353803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33e0c-fd43-4d47-bf35-41401c28b5d5 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:13.354371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33e0c-fd43-4d47-bf35-41401c28b5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:13.354371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33e0c-fd43-4d47-bf35-41401c28b5d5 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:13.355607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:13.362166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61873e7c-86eb-4062-8ee2-ea20cfb9b1d4 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:13.362713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61873e7c-86eb-4062-8ee2-ea20cfb9b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:13.362877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61873e7c-86eb-4062-8ee2-ea20cfb9b1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:13.432576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-61873e7c-86eb-4062-8ee2-ea20cfb9b1d4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:13.433855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-61873e7c-86eb-4062-8ee2-ea20cfb9b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:13.452964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:13.535619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:13.819945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c33e0c-fd43-4d47-bf35-41401c28b5d5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:13.821475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b6c33e0c-fd43-4d47-bf35-41401c28b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:13.939287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c33e0c-fd43-4d47-bf35-41401c28b5d5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:14.007842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:14.015172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:14.015802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:14.016073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:14.016330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:14.017913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:14.131651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:14.133096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8848f-6566-4bfd-9c08-b52c7258b519 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:14.151398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 742/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:16.373974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf80c7c-4fae-4ce3-b334-f56c9d23d5e3 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:16.374792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf80c7c-4fae-4ce3-b334-f56c9d23d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:16.374953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf80c7c-4fae-4ce3-b334-f56c9d23d5e3 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:16.376610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:16.382559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e62eb7-623b-4ef1-a6f1-ac6a4393baff None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:16.383032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e62eb7-623b-4ef1-a6f1-ac6a4393baff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:16.383146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e62eb7-623b-4ef1-a6f1-ac6a4393baff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:16.450455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-38e62eb7-623b-4ef1-a6f1-ac6a4393baff None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:16.451708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-38e62eb7-623b-4ef1-a6f1-ac6a4393baff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:16.504837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:16.554995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:16.841378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf80c7c-4fae-4ce3-b334-f56c9d23d5e3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:16.842996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4bf80c7c-4fae-4ce3-b334-f56c9d23d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:16.953144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf80c7c-4fae-4ce3-b334-f56c9d23d5e3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:17.003137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:17.029929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:17.030537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:17.030817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:17.031087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:17.033124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:17.147825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:17.149282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d9cff-f280-4447-8bf4-8a44d14ce837 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:17.167732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 727/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:26.557708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=22 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:27.165391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=66 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:37.433633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9096e-52a0-492b-a766-a326d95f6e92 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:37.434200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9096e-52a0-492b-a766-a326d95f6e92 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:37.434332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9096e-52a0-492b-a766-a326d95f6e92 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:37.436020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 744/1471] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:37.442366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9db9c-a02d-4e7a-8554-030e720466c5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:37.442814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9db9c-a02d-4e7a-8554-030e720466c5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:37.442923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9db9c-a02d-4e7a-8554-030e720466c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:37.513392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-30c9db9c-a02d-4e7a-8554-030e720466c5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:37.515304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-30c9db9c-a02d-4e7a-8554-030e720466c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:37.613981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 728/1472] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:37.913627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab90a-bd48-4079-9280-f81fb16b5485 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:37.913966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab90a-bd48-4079-9280-f81fb16b5485 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:37.914235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab90a-bd48-4079-9280-f81fb16b5485 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:37.915613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 745/1473] 10.222.0.55 () {54 vars in 962 bytes} [Sat Sep 24 19:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:37.920625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61ca1c-5c85-4a9d-a36a-a8f28875452f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:37.921038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61ca1c-5c85-4a9d-a36a-a8f28875452f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:37.921416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61ca1c-5c85-4a9d-a36a-a8f28875452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:38.002593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6e61ca1c-5c85-4a9d-a36a-a8f28875452f None None] MFA Rules not processed for user `6e9ce9098e884f4e9119aeccdbafd4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:38.003765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6e61ca1c-5c85-4a9d-a36a-a8f28875452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:38.026864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:38.091978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 729/1474] 10.222.0.55 () {58 vars in 1059 bytes} [Sat Sep 24 19:39:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:38.100358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ab90a-bd48-4079-9280-f81fb16b5485 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:38.101880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a59ab90a-bd48-4079-9280-f81fb16b5485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:38.208892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ab90a-bd48-4079-9280-f81fb16b5485 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:38.281524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:38.282149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:38.282421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:38.282683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:38.284291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:38.338807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:38.394663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:38.396100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1d01e-57d3-4447-88f8-e9f2f987744a service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:38.413883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 746/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:40.673660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a521ccf-1493-4201-ab61-78775026522a None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:40.674275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a521ccf-1493-4201-ab61-78775026522a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:40.674275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a521ccf-1493-4201-ab61-78775026522a None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:40.675722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 730/1476] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:40.681943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd1e4-54a2-4730-a129-03b631c1601b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:40.682358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd1e4-54a2-4730-a129-03b631c1601b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:40.682485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bd1e4-54a2-4730-a129-03b631c1601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:40.754824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0b2bd1e4-54a2-4730-a129-03b631c1601b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:40.756064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0b2bd1e4-54a2-4730-a129-03b631c1601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:40.851863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 747/1477] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:41.145908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a521ccf-1493-4201-ab61-78775026522a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:41.147579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8a521ccf-1493-4201-ab61-78775026522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:41.277271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a521ccf-1493-4201-ab61-78775026522a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:41.326007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:41.370124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:41.370719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:41.370990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:41.371253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:41.371911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:41.373815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:39:41.373815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:39:41.373815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:39:41.374946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289cd68-4dc2-4b40-a2b2-0e2e1e0e0dbf admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:41.394474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 731/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Sat Sep 24 19:39:41 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:39:43.565170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486cfc0-0a7c-4705-a4c1-f96ffbc72f0f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:43.565723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486cfc0-0a7c-4705-a4c1-f96ffbc72f0f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:43.565723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486cfc0-0a7c-4705-a4c1-f96ffbc72f0f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:43.567044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 748/1479] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:43.572357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528a70e-48eb-4146-9e1a-49c75bcd7f05 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:43.572822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528a70e-48eb-4146-9e1a-49c75bcd7f05 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:43.572936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528a70e-48eb-4146-9e1a-49c75bcd7f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:43.639793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9528a70e-48eb-4146-9e1a-49c75bcd7f05 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:43.641004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9528a70e-48eb-4146-9e1a-49c75bcd7f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:43.736232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 732/1480] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:44.032490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486cfc0-0a7c-4705-a4c1-f96ffbc72f0f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:44.034060 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d486cfc0-0a7c-4705-a4c1-f96ffbc72f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:44.043764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:44.181201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486cfc0-0a7c-4705-a4c1-f96ffbc72f0f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:44.261240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:44.294120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:44.294656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:44.295512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:44.295780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:44.297388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:44.387867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:44.389488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3f8b2-993f-4785-8d40-d2d525774f8e service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:44.408084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 749/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:46.706957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18404e53-ad3c-4320-a638-5bc63fc45ed3 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:46.707557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18404e53-ad3c-4320-a638-5bc63fc45ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:46.707557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18404e53-ad3c-4320-a638-5bc63fc45ed3 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:46.708891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 733/1482] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:46.714367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5b0f-a96d-4fd6-a6b5-e52c7f0589c0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:46.714802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5b0f-a96d-4fd6-a6b5-e52c7f0589c0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:46.714916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5b0f-a96d-4fd6-a6b5-e52c7f0589c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:46.782969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-04ad5b0f-a96d-4fd6-a6b5-e52c7f0589c0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:46.784228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-04ad5b0f-a96d-4fd6-a6b5-e52c7f0589c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:46.832237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:46.882849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 750/1483] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:47.172630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18404e53-ad3c-4320-a638-5bc63fc45ed3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:47.174175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-18404e53-ad3c-4320-a638-5bc63fc45ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:47.197839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:47.277372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18404e53-ad3c-4320-a638-5bc63fc45ed3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:47.350347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:47.350868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:47.351143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:47.351409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:47.353019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:47.426068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:47.461788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:47.463227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9df07-9cc0-4b9a-ab00-44e6a76cf1cb service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:47.482969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 734/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:49.684472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6cd68-9e3b-4252-9103-6afcb512ed94 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:49.685583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6cd68-9e3b-4252-9103-6afcb512ed94 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:49.685583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6cd68-9e3b-4252-9103-6afcb512ed94 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:49.687307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 751/1485] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:49.692508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22790ce1-88f0-455a-9f56-0fac34ed2dc7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:49.692947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22790ce1-88f0-455a-9f56-0fac34ed2dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:49.693072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22790ce1-88f0-455a-9f56-0fac34ed2dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:49.757844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-22790ce1-88f0-455a-9f56-0fac34ed2dc7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:49.759040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-22790ce1-88f0-455a-9f56-0fac34ed2dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:49.856506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 735/1486] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:50.144248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6cd68-9e3b-4252-9103-6afcb512ed94 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:50.145781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-29f6cd68-9e3b-4252-9103-6afcb512ed94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:50.250914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6cd68-9e3b-4252-9103-6afcb512ed94 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:50.322557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:50.324230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:50.324879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:50.325394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:50.325742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:50.327883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:50.436400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:50.437815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a0a70-18e1-46c7-a409-69d852ac177d service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:50.455484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 752/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:52.833930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f95f2-aab4-44dc-8648-4e178d70283d None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:52.834527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f95f2-aab4-44dc-8648-4e178d70283d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:52.834527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f95f2-aab4-44dc-8648-4e178d70283d None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:52.835849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 736/1488] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:39:52.841721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1f97d-aeb6-48d7-a921-74d33935b05a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:52.842239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1f97d-aeb6-48d7-a921-74d33935b05a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:52.842518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1f97d-aeb6-48d7-a921-74d33935b05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:52.908753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2fc1f97d-aeb6-48d7-a921-74d33935b05a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:52.909963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2fc1f97d-aeb6-48d7-a921-74d33935b05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:52.962869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:53.007242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 753/1489] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:53.295457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f95f2-aab4-44dc-8648-4e178d70283d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:53.297006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-992f95f2-aab4-44dc-8648-4e178d70283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:53.333219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:53.400142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f95f2-aab4-44dc-8648-4e178d70283d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:53.472411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:53.473004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:53.473279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:53.473544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:53.475104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:53.553200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:53.583086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:53.584572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b5d66-5b5c-40f3-af67-b992b2787665 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:53.602244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 737/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:39:55.975346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e775d6a-abde-4d86-9629-e20deb4294ca None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:55.975968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e775d6a-abde-4d86-9629-e20deb4294ca None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:55.975968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e775d6a-abde-4d86-9629-e20deb4294ca None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:55.977275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 754/1491] 10.222.0.55 () {54 vars in 927 bytes} [Sat Sep 24 19:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:39:55.983776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63799ad3-b448-4604-9995-993a92372a53 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:55.984294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63799ad3-b448-4604-9995-993a92372a53 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:55.984294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63799ad3-b448-4604-9995-993a92372a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:56.057590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-63799ad3-b448-4604-9995-993a92372a53 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:39:56.058814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-63799ad3-b448-4604-9995-993a92372a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:56.160088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 738/1492] 10.222.0.55 () {58 vars in 1024 bytes} [Sat Sep 24 19:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:39:56.460926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e775d6a-abde-4d86-9629-e20deb4294ca None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:39:56.462468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4e775d6a-abde-4d86-9629-e20deb4294ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:56.572939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e775d6a-abde-4d86-9629-e20deb4294ca None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:39:56.648334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:39:56.648935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:39:56.649213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:39:56.649484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:39:56.651082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:39:56.660603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:39:56.767039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:39:56.768687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dcf5f-eb27-4ed7-a4c3-3be4280e5280 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:39:56.789933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 755/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:40:06.158428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=82 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:40:06.787615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=50 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:10.479684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:10.480350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:10.480350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:10.558476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:41:10.559738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:10.657905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 739/1494] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:41:10.666099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:10.667325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:10.667325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:10.742819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:41:10.743976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:10.844095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 756/1495] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:41:10.848912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:10.850242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:10.861969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de30192-aba8-46d7-adc6-d4261800b907 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:10.924609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:10.959160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:10.959797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:10.960082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:10.960378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:10.961033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:41:10.961314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:10.962328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:10.962328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:41:10.962328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:41:10.963339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:10.974724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 740/1496] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:41:10.984193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:10.993356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d89b-4e1f-4e42-a7f1-3badf253e073 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:11.055386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:11.091028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:11.091692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:11.091971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:11.092240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:11.092863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:11.094603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:11.094603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:41:11.094603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:41:11.095861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:11.154147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 757/1497] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:41:10 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:41:11.158525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:11.160091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:11.194374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd17433-0370-4400-9c6a-debe31066c9b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:11.295334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:11.295884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:11.296160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:11.296426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:11.297050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:11.298066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:11.298066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:41:11.298066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:41:11.299573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:11.348074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 741/1498] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:41:11 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:41:11.352575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:11.354691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:11.420605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611752-99b9-455a-8d68-8d1639fe74ac admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:11.510609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:11.526449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:11.527062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:11.527333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:11.527644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:11.528286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:11.529288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:11.529288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:41:11.529288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:41:11.530189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:11.544974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 758/1499] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:41:11.549366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:11.550868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:11.562577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff162e7-a558-44e8-afed-1f9f76ccf96c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:11.587739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:11.660496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:11.661106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:11.661378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:11.661642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:11.662398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:11.663410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:11.663410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:41:11.663410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:41:11.664338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5925d3-1ad5-4477-9691-0286a30b07e5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:11.677320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 742/1500] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:41:11.681688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:11.683657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:11.720144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04bf9-6986-497e-bbdb-ebd90d5ddddd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:11.815063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:11.815665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:11.815940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:11.816203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] PATH_INFO: `/v3/projects/0c91d3000d194164ac04737a70260160/users/ef9b9247037d410688a3ea1452689611/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:11.840631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c91d3000d194164ac04737a70260160, user_id=ef9b9247037d410688a3ea1452689611, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:11.841667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:11.841667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:41:11.841667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:41:11.843646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5e6db-61dc-4754-846a-89827f737308 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:11.869316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 759/1501] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:41:11 2022] PUT /identity/v3/projects/0c91d3000d194164ac04737a70260160/users/ef9b9247037d410688a3ea1452689611/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:41:11.882199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ac2d6-5e30-4afa-b846-ce7760a9e1e6 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:11.882494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ac2d6-5e30-4afa-b846-ce7760a9e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:11.882824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ac2d6-5e30-4afa-b846-ce7760a9e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:11.946226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-976ac2d6-5e30-4afa-b846-ce7760a9e1e6 None None] MFA Rules not processed for user `ef9b9247037d410688a3ea1452689611`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:41:11.947965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-976ac2d6-5e30-4afa-b846-ce7760a9e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:12.026400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 743/1502] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:41:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:41:12.039704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83742f-0aee-489e-afc1-b16ad5b55647 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:12.039984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83742f-0aee-489e-afc1-b16ad5b55647 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:12.040277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83742f-0aee-489e-afc1-b16ad5b55647 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:12.041784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 760/1503] 10.222.0.55 () {54 vars in 963 bytes} [Sat Sep 24 19:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:41:12.046836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958698cf-e4fa-449c-a934-0ba0e0bbb3b7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:12.047116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958698cf-e4fa-449c-a934-0ba0e0bbb3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:12.047412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958698cf-e4fa-449c-a934-0ba0e0bbb3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:12.081863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:12.126979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-958698cf-e4fa-449c-a934-0ba0e0bbb3b7 None None] MFA Rules not processed for user `b93ea821819d462097b1e76ce1ffae3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:41:12.128284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-958698cf-e4fa-449c-a934-0ba0e0bbb3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:12.206868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 744/1504] 10.222.0.55 () {58 vars in 1060 bytes} [Sat Sep 24 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:41:12.216758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83742f-0aee-489e-afc1-b16ad5b55647 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:12.218277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cd83742f-0aee-489e-afc1-b16ad5b55647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:12.268990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:12.325333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83742f-0aee-489e-afc1-b16ad5b55647 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:12.400593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:12.401321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:12.401599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:12.401866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:12.403561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:12.504075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:12.505508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:12.508943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:12.524015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 761/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:41:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:41:12.570146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae26839-b9d5-4f31-932b-64f0b99d9141 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:12.570585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae26839-b9d5-4f31-932b-64f0b99d9141 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:12.570682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae26839-b9d5-4f31-932b-64f0b99d9141 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:12.571971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 745/1506] 10.222.0.55 () {52 vars in 830 bytes} [Sat Sep 24 19:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:41:12.575650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:12.576854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:12.674843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5ecd7-9304-4cc8-98d2-f0036af68d2f service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:12.767607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:12.768433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:12.768433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:12.768706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] PATH_INFO: `/v3/users/ef9b9247037d410688a3ea1452689611` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:12.781700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9b9247037d410688a3ea1452689611)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:12.782709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:12.782709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:41:12.782709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:41:12.783947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:12.793930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:12.851660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9b9247037d410688a3ea1452689611'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:12.876052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9b9247037d410688a3ea1452689611'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:12.885021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9b9247037d410688a3ea1452689611'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:12.885596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9b9247037d410688a3ea1452689611'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:12.899954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9b9247037d410688a3ea1452689611'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:12.906161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 762/1507] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:41:12 2022] DELETE /identity/v3/users/ef9b9247037d410688a3ea1452689611 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:41:12.910749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:12.911036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:12.911311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:12.969624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:41:12.970749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:13.045594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:13.058918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 746/1508] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:41:13.068663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:13.069898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:13.170965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cceee1-bad1-418c-9acc-d336b7b4da5c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:13.225064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:13.244706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:13.245395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:13.245669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:13.245933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:13.247535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:13.355749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:13.357178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ee263-cf1c-4f60-8116-b7f7f1ca8159 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:13.376076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 763/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:41:13.780008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:41:13.781312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:41:13.794304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533c50d-6a9c-43eb-a898-c2db41e877e7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:41:13.905237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:41:13.905868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:41:13.906150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:41:13.906412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] PATH_INFO: `/v3/projects/0c91d3000d194164ac04737a70260160` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:41:13.917591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c91d3000d194164ac04737a70260160)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:41:13.918830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:41:13.918830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:41:13.918830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:41:13.920130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:41:13.983535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c91d3000d194164ac04737a70260160', 'request_id': 'req-6c057c38-3d14-4247-b9af-a15855de7411'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:13.995311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c91d3000d194164ac04737a70260160', 'request_id': 'req-6c057c38-3d14-4247-b9af-a15855de7411'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:14.001821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c91d3000d194164ac04737a70260160', 'request_id': 'req-6c057c38-3d14-4247-b9af-a15855de7411'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:14.032310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] The token cache is being invalidate because project 0c91d3000d194164ac04737a70260160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:41:14.032524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:14.032920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c057c38-3d14-4247-b9af-a15855de7411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:41:14.034364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 747/1510] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:41:13 2022] DELETE /identity/v3/projects/0c91d3000d194164ac04737a70260160 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:41:23.373823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=62 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:41:24.030159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:42:44.506260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6bdd3-939d-43f1-a3a6-a6cdf9eed6af None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:44.507001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6bdd3-939d-43f1-a3a6-a6cdf9eed6af None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:44.507001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6bdd3-939d-43f1-a3a6-a6cdf9eed6af None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:44.508225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:42:44.513230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08916ca-bc54-4a22-b30f-660550ea5dfb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:44.513653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08916ca-bc54-4a22-b30f-660550ea5dfb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:44.513754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08916ca-bc54-4a22-b30f-660550ea5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:44.596798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-d08916ca-bc54-4a22-b30f-660550ea5dfb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:42:44.598050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d08916ca-bc54-4a22-b30f-660550ea5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:44.706067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:42:44.862239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6bdd3-939d-43f1-a3a6-a6cdf9eed6af None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:42:44.864367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-48a6bdd3-939d-43f1-a3a6-a6cdf9eed6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:44.983562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6bdd3-939d-43f1-a3a6-a6cdf9eed6af None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:42:44.989082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41fd80-3c3e-4efc-8621-f457ed4b0b72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:42:44.989686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41fd80-3c3e-4efc-8621-f457ed4b0b72 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:44.989961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41fd80-3c3e-4efc-8621-f457ed4b0b72 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:44.990227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41fd80-3c3e-4efc-8621-f457ed4b0b72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:44.999165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41fd80-3c3e-4efc-8621-f457ed4b0b72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:42:45.000786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41fd80-3c3e-4efc-8621-f457ed4b0b72 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:42:45.011383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 765/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:42:45.996172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0111dd9-4a70-47b6-b677-2f2cda133848 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:45.996758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0111dd9-4a70-47b6-b677-2f2cda133848 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:45.996758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0111dd9-4a70-47b6-b677-2f2cda133848 None None] PATH_INFO: `/v3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:45.998026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 749/1514] 10.222.0.65 () {54 vars in 925 bytes} [Sat Sep 24 19:42:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:42:46.004901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe346013-2309-4610-b689-992ed5625388 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:46.005333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe346013-2309-4610-b689-992ed5625388 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:46.005448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe346013-2309-4610-b689-992ed5625388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:46.076089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fe346013-2309-4610-b689-992ed5625388 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:42:46.077487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fe346013-2309-4610-b689-992ed5625388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:46.132984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2253-9996-40c4-8808-cbcbbcb9b727 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:46.133267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2253-9996-40c4-8808-cbcbbcb9b727 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:46.133538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2253-9996-40c4-8808-cbcbbcb9b727 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:46.135057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 750/1515] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:42:46.141267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78afaec-bd00-4656-9610-a7aee260838c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:46.141547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78afaec-bd00-4656-9610-a7aee260838c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:46.141818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78afaec-bd00-4656-9610-a7aee260838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:46.148877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:42:46.154205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 766/1516] 10.222.0.65 () {58 vars in 1015 bytes} [Sat Sep 24 19:42:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:42:46.162924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c78afaec-bd00-4656-9610-a7aee260838c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:42:46.164328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c78afaec-bd00-4656-9610-a7aee260838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:46.172601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 751/1517] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:42:46.330121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe346013-2309-4610-b689-992ed5625388 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:42:46.342272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe346013-2309-4610-b689-992ed5625388 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:42:46.347675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990a428-ee42-4c7f-b393-97a9491e11ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:42:46.348277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990a428-ee42-4c7f-b393-97a9491e11ea service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:46.348545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990a428-ee42-4c7f-b393-97a9491e11ea service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:46.348804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990a428-ee42-4c7f-b393-97a9491e11ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:46.357994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990a428-ee42-4c7f-b393-97a9491e11ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:42:46.359627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990a428-ee42-4c7f-b393-97a9491e11ea service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:42:46.370163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 767/1518] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:42:54.952102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed7cf-64fe-4ed9-8bfd-b2f8a8cec503 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:54.952641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed7cf-64fe-4ed9-8bfd-b2f8a8cec503 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:54.952641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed7cf-64fe-4ed9-8bfd-b2f8a8cec503 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:54.953981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 752/1519] 10.222.0.55 () {54 vars in 961 bytes} [Sat Sep 24 19:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:42:54.959457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1721b-f1d0-451e-9d9c-f28bb009cc5c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:54.960227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1721b-f1d0-451e-9d9c-f28bb009cc5c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:54.960494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1721b-f1d0-451e-9d9c-f28bb009cc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:54.980033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3ab1721b-f1d0-451e-9d9c-f28bb009cc5c None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:42:54.981235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3ab1721b-f1d0-451e-9d9c-f28bb009cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:54.989648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 768/1520] 10.222.0.55 () {58 vars in 1058 bytes} [Sat Sep 24 19:42:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:42:54.997151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ed7cf-64fe-4ed9-8bfd-b2f8a8cec503 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:42:55.010710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ed7cf-64fe-4ed9-8bfd-b2f8a8cec503 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:42:55.015647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87df44-6683-4303-9739-4c9c6d4c0755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:42:55.016243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87df44-6683-4303-9739-4c9c6d4c0755 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:55.016516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87df44-6683-4303-9739-4c9c6d4c0755 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:55.016782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87df44-6683-4303-9739-4c9c6d4c0755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:55.025875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87df44-6683-4303-9739-4c9c6d4c0755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:42:55.027494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87df44-6683-4303-9739-4c9c6d4c0755 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:42:55.037569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 753/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:42:57.849575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022abd7c-83e9-41b7-bbf5-c647a2c1f722 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:57.850363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022abd7c-83e9-41b7-bbf5-c647a2c1f722 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:57.850363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022abd7c-83e9-41b7-bbf5-c647a2c1f722 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:57.851502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:42:57.857851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762be9e-825f-44c3-8aeb-ace8b7fbd4d8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:57.858357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762be9e-825f-44c3-8aeb-ace8b7fbd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:57.858357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762be9e-825f-44c3-8aeb-ace8b7fbd4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:57.878915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6762be9e-825f-44c3-8aeb-ace8b7fbd4d8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:42:57.880136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6762be9e-825f-44c3-8aeb-ace8b7fbd4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:57.889004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:42:58.048159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022abd7c-83e9-41b7-bbf5-c647a2c1f722 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:42:58.063873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022abd7c-83e9-41b7-bbf5-c647a2c1f722 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:42:58.070008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576b23-f84e-4907-8832-f793a55d759f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:42:58.070601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576b23-f84e-4907-8832-f793a55d759f service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:58.070875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576b23-f84e-4907-8832-f793a55d759f service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:58.071138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576b23-f84e-4907-8832-f793a55d759f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:58.079584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576b23-f84e-4907-8832-f793a55d759f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:42:58.081149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576b23-f84e-4907-8832-f793a55d759f service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:42:58.090547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 770/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:42:59.167183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c78ce-a17a-4aa3-907f-09f744e9c2b2 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:59.167781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c78ce-a17a-4aa3-907f-09f744e9c2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:59.167781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c78ce-a17a-4aa3-907f-09f744e9c2b2 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:59.169095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 19:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:42:59.175663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d90104-76db-4813-a5d7-a774a1ec1300 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:59.176110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d90104-76db-4813-a5d7-a774a1ec1300 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:59.176228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d90104-76db-4813-a5d7-a774a1ec1300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:59.196138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b6d90104-76db-4813-a5d7-a774a1ec1300 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:42:59.197351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b6d90104-76db-4813-a5d7-a774a1ec1300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:42:59.205275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 19:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:42:59.358973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c78ce-a17a-4aa3-907f-09f744e9c2b2 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:42:59.374372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c78ce-a17a-4aa3-907f-09f744e9c2b2 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:42:59.384175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e86047-8dfd-4233-a2f9-bdb72a58fa78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:42:59.385216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e86047-8dfd-4233-a2f9-bdb72a58fa78 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:42:59.385716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e86047-8dfd-4233-a2f9-bdb72a58fa78 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:42:59.386193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e86047-8dfd-4233-a2f9-bdb72a58fa78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:42:59.395105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e86047-8dfd-4233-a2f9-bdb72a58fa78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:42:59.396792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e86047-8dfd-4233-a2f9-bdb72a58fa78 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:42:59.406182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 756/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:43:03.641933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d01f7-fb9d-4bec-9a83-ab378a4e8ec7 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:03.642511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d01f7-fb9d-4bec-9a83-ab378a4e8ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:03.642511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d01f7-fb9d-4bec-9a83-ab378a4e8ec7 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:03.643841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:43:03.649046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191c0cc-a085-4cb7-a767-decb0b49af72 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:03.649476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191c0cc-a085-4cb7-a767-decb0b49af72 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:03.649586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191c0cc-a085-4cb7-a767-decb0b49af72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:03.670698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6191c0cc-a085-4cb7-a767-decb0b49af72 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:43:03.672226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6191c0cc-a085-4cb7-a767-decb0b49af72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:03.680972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:43:03.967880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d01f7-fb9d-4bec-9a83-ab378a4e8ec7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:43:03.969673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd5d01f7-fb9d-4bec-9a83-ab378a4e8ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:04.047598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d01f7-fb9d-4bec-9a83-ab378a4e8ec7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:43:04.051164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d4204-0e22-4bff-a042-53fafe276197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:43:04.051797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4204-0e22-4bff-a042-53fafe276197 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:04.052072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4204-0e22-4bff-a042-53fafe276197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:04.052361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4204-0e22-4bff-a042-53fafe276197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:04.061776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4204-0e22-4bff-a042-53fafe276197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:43:04.063219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4204-0e22-4bff-a042-53fafe276197 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:43:04.073163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 773/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:43:06.226977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a40ba-b8a7-4e1d-8b29-31c2e744c0ca None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:06.228943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a40ba-b8a7-4e1d-8b29-31c2e744c0ca None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:06.228943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a40ba-b8a7-4e1d-8b29-31c2e744c0ca None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:06.229146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:43:06.234613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55262e7e-8dbf-4b82-b179-6daf36a17217 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:06.235110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55262e7e-8dbf-4b82-b179-6daf36a17217 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:06.235214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55262e7e-8dbf-4b82-b179-6daf36a17217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:06.255782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-55262e7e-8dbf-4b82-b179-6daf36a17217 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:43:06.257033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-55262e7e-8dbf-4b82-b179-6daf36a17217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:06.264931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:43:06.551151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a40ba-b8a7-4e1d-8b29-31c2e744c0ca None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:43:06.563024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a40ba-b8a7-4e1d-8b29-31c2e744c0ca None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:43:06.566581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b062-e994-4b3f-8f45-75b56bf37f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:43:06.567183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b062-e994-4b3f-8f45-75b56bf37f42 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:06.567496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b062-e994-4b3f-8f45-75b56bf37f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:06.568307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b062-e994-4b3f-8f45-75b56bf37f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:06.577434 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b062-e994-4b3f-8f45-75b56bf37f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:43:06.578874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b062-e994-4b3f-8f45-75b56bf37f42 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:43:06.590912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 759/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:43:10.353430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab2ea3-2996-4e35-9feb-3e79d7e91b49 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:10.353982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab2ea3-2996-4e35-9feb-3e79d7e91b49 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:10.353982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab2ea3-2996-4e35-9feb-3e79d7e91b49 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:10.355213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:43:10.360684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ce6a-3f17-4ef4-8119-c6b9801faf56 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:10.361111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ce6a-3f17-4ef4-8119-c6b9801faf56 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:10.361303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ce6a-3f17-4ef4-8119-c6b9801faf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:10.382184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9080ce6a-3f17-4ef4-8119-c6b9801faf56 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:43:10.383388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9080ce6a-3f17-4ef4-8119-c6b9801faf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:10.391211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:43:10.696661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab2ea3-2996-4e35-9feb-3e79d7e91b49 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:43:10.698348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2eab2ea3-2996-4e35-9feb-3e79d7e91b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:10.777829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab2ea3-2996-4e35-9feb-3e79d7e91b49 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:43:10.781941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b720ba0-cce3-4156-9a90-629e327f466a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:43:10.782552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b720ba0-cce3-4156-9a90-629e327f466a service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:10.782833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b720ba0-cce3-4156-9a90-629e327f466a service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:10.783105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b720ba0-cce3-4156-9a90-629e327f466a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:10.792632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b720ba0-cce3-4156-9a90-629e327f466a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:43:10.794057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b720ba0-cce3-4156-9a90-629e327f466a service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:43:10.804362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 776/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:43:12.961680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55844c-1ff1-4e0a-923c-221b3e85fbc7 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:12.962329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55844c-1ff1-4e0a-923c-221b3e85fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:12.962329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55844c-1ff1-4e0a-923c-221b3e85fbc7 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:12.963650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 19:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:43:12.969806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159b73c-3e68-4873-8dd2-f7a369df41c5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:12.970360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159b73c-3e68-4873-8dd2-f7a369df41c5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:12.970360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159b73c-3e68-4873-8dd2-f7a369df41c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:12.990799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3159b73c-3e68-4873-8dd2-f7a369df41c5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:43:12.992041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3159b73c-3e68-4873-8dd2-f7a369df41c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:12.999745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 19:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:43:13.298708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55844c-1ff1-4e0a-923c-221b3e85fbc7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:43:13.300397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bc55844c-1ff1-4e0a-923c-221b3e85fbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:43:13.308733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:43:13.315025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55844c-1ff1-4e0a-923c-221b3e85fbc7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:43:13.318716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd7b31-c1f0-4294-b0b9-f322230ed347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:43:13.319308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7b31-c1f0-4294-b0b9-f322230ed347 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:43:13.319613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7b31-c1f0-4294-b0b9-f322230ed347 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:43:13.319886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7b31-c1f0-4294-b0b9-f322230ed347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:43:13.328814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7b31-c1f0-4294-b0b9-f322230ed347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:43:13.330270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7b31-c1f0-4294-b0b9-f322230ed347 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:43:13.340051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 762/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:43:22.985207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:43:23.335817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:38.805574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e8517-87eb-48d1-a15d-9db5fc4c519c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:38.806241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e8517-87eb-48d1-a15d-9db5fc4c519c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:38.806241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e8517-87eb-48d1-a15d-9db5fc4c519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:38.840591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a07e8517-87eb-48d1-a15d-9db5fc4c519c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:38.842005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a07e8517-87eb-48d1-a15d-9db5fc4c519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:38.853178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 778/1540] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:38.862567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:38.863180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:38.863180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:38.876100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1eeec-60df-4efb-a963-3d329764a321 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:38.876611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1eeec-60df-4efb-a963-3d329764a321 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:38.876959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1eeec-60df-4efb-a963-3d329764a321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:38.890615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:38.892520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:38.909563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 763/1541] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:38.909787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5cb1eeec-60df-4efb-a963-3d329764a321 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:38.911389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5cb1eeec-60df-4efb-a963-3d329764a321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:38.914353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:38.925249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43798c7-2803-46bd-8725-9615f7507559 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:38.933288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 779/1542] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:38.941457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:38.942093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:38.942372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:38.942641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:38.943294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:38.943818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:38.945802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dedd59-329d-4511-a55e-b2bed9f6ddec admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:38.946600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:38.946892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:38.947166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:38.965791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 764/1543] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:38.966882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:38.967166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:38.967454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:38.981354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:38.982579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.000205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 780/1544] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.001113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.008806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.010046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.018421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63b2b0-855f-4e08-8dc0-1597ddf5e08d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.029716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46679ac-33cd-4bee-ae92-a62d7b0356a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.030354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46679ac-33cd-4bee-ae92-a62d7b0356a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.030631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46679ac-33cd-4bee-ae92-a62d7b0356a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.030897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46679ac-33cd-4bee-ae92-a62d7b0356a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.031579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46679ac-33cd-4bee-ae92-a62d7b0356a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.034388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 765/1545] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.034538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46679ac-33cd-4bee-ae92-a62d7b0356a0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.034942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.059636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a092c36-eb3b-4964-8e7f-747bfc3440a6 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.085631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.086952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.087535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.088124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.089135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.089653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.091673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e56b7-44a8-41da-bd94-5c16a31e0d17 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.107666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 781/1546] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.108982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c665aa-9d39-4423-8b0e-b4690412653a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.109269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c665aa-9d39-4423-8b0e-b4690412653a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.109536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c665aa-9d39-4423-8b0e-b4690412653a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.116777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 766/1547] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.118040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.118320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.118587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.149881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-57c665aa-9d39-4423-8b0e-b4690412653a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.152230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-57c665aa-9d39-4423-8b0e-b4690412653a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.161718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.166287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.257279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 782/1548] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.258901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.259222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 767/1549] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.259612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.260080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.260235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.271667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a957b-e98c-40d8-8f86-01ee1440bb9a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.276438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e71838-b4b5-4571-bbbe-1bab3d32b1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.277046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e71838-b4b5-4571-bbbe-1bab3d32b1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.277324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e71838-b4b5-4571-bbbe-1bab3d32b1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.277590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e71838-b4b5-4571-bbbe-1bab3d32b1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.278217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e71838-b4b5-4571-bbbe-1bab3d32b1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.280180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e71838-b4b5-4571-bbbe-1bab3d32b1f6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.281007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.282230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.290269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 783/1550] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.297807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.301965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473d151-031d-4f08-b4ca-eeb5bebbcb23 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.303162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 768/1551] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.304435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc93de-e4ee-4b6c-9c8b-bd6e933038fd None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.304720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc93de-e4ee-4b6c-9c8b-bd6e933038fd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.304988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc93de-e4ee-4b6c-9c8b-bd6e933038fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.307310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dff74b-7bfe-49e0-a1e7-0c178a4956e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.307963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff74b-7bfe-49e0-a1e7-0c178a4956e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.308243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff74b-7bfe-49e0-a1e7-0c178a4956e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.309142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff74b-7bfe-49e0-a1e7-0c178a4956e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.309904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dff74b-7bfe-49e0-a1e7-0c178a4956e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.311959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dff74b-7bfe-49e0-a1e7-0c178a4956e7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.324612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-65fc93de-e4ee-4b6c-9c8b-bd6e933038fd None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.325811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-65fc93de-e4ee-4b6c-9c8b-bd6e933038fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.335031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 769/1552] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.336072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.336351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.336617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.349654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 784/1553] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.351174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1061bd7-9cfe-4727-a212-6b1d88f93d31 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.351538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1061bd7-9cfe-4727-a212-6b1d88f93d31 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.351821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1061bd7-9cfe-4727-a212-6b1d88f93d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.357286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.358491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.371833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d1061bd7-9cfe-4727-a212-6b1d88f93d31 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.373029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d1061bd7-9cfe-4727-a212-6b1d88f93d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.419877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 770/1554] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.421001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.430296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb3adb-a1f0-453d-a2a6-02da3a1c9c1a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.434411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 785/1555] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.435175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.435610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.435801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.435931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.436152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.436266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.436346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.436969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.437268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.439204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ad67c-6f08-41fa-a98d-c43bab3ea252 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.455516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 771/1556] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.455516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af505fcd-f5bf-46af-8f4b-c275d57e808a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.455516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af505fcd-f5bf-46af-8f4b-c275d57e808a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.455516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af505fcd-f5bf-46af-8f4b-c275d57e808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.458305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.459572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.467825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 786/1557] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.467825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.477219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-af505fcd-f5bf-46af-8f4b-c275d57e808a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.478549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-af505fcd-f5bf-46af-8f4b-c275d57e808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.478794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9d72a-d38d-4335-b63f-4a897d7dcd2a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.483361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.483983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.484257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.484530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.485169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.486698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 772/1558] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.486955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.487971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae68a2c-6119-46dc-98ea-d903e5bb1cc9 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.488458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae68a2c-6119-46dc-98ea-d903e5bb1cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.488733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae68a2c-6119-46dc-98ea-d903e5bb1cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.497485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 787/1559] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:39.498343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.505204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:39.508926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8ae68a2c-6119-46dc-98ea-d903e5bb1cc9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.509892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa30f9-53eb-4858-8ee4-e668572016bd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.510136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8ae68a2c-6119-46dc-98ea-d903e5bb1cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.516283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.517537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.517844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.518160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.518835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 773/1560] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.518988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.520073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:44:39.520073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:44:39.520073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:44:39.520354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.520443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.520642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.521505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546863f4-3320-4cda-84e3-78e0419e48b3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.541055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.542256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.547460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 788/1561] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.548807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.549138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.549451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.551369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 774/1562] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.552184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.558156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:39.562221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda252-8a88-4019-91a7-a886c805aea5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.568171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.568777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.569054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.569321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.569949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.571021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:44:39.571021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:44:39.571021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:44:39.572255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.572382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:39.573536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:39.582548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 789/1563] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:39.583320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.592644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d8604-6e86-4872-ae5f-ea2bf914d2fe None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.599061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.599696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.599974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.600238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.600873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.601170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.602203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:44:39.602203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:44:39.602203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:44:39.603111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.607637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 775/1564] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.607637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.622159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 790/1565] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.622159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.625434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa12738-ce95-4c93-911a-683b04ab847f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.681469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd77e5-a841-4ba1-8fbc-0d44bc010d81 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.684357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.685152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.685434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.685713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.686350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.687202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.687931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.688125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.688373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.688840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.689693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.690146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.691884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.698703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 791/1566] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:39.699589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.707226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 776/1567] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.714495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.714599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fe381-ec39-445d-b689-5ed0887f4c88 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.716977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.717562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.717838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.718213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.718852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.719145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.721526 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.723924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ecae-74b7-4bb6-8b62-0f42f382b187 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.728554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.729149 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.729425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.729694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.730329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.731947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.734191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 792/1568] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.734995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.744556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb028-da05-4668-bf7b-1ef58c1826b5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.748028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 777/1569] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:39.748028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.749257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.749847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.750124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.750395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.751567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.751873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.753473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.757646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8f50a-708b-4da9-8b7e-cdd79bb1e183 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.762198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.762776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.763052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.763314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.763970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:39.764269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.764917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 793/1570] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.765765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.765892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.785393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 778/1571] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:39.785393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.785722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2c17-b665-4449-8dba-a9c4b6479075 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.787097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.787805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.788095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.788359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.789053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.790938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.791934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6409b5b-128a-4381-a93f-b9aec875bc44 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.796809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.797397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.797670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.797961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.798745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.800680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.815305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 794/1572] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.816105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.825814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa323-3380-4e98-8653-89abf17586f1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.830574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.830946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 779/1573] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.831222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.831530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.831778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.831879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] PATH_INFO: `/v3/projects/977038321e5a41e8ab137fede4d118d7/users/444719e34bf2457b9a58e182ef7fbd79/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.847510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=977038321e5a41e8ab137fede4d118d7, user_id=444719e34bf2457b9a58e182ef7fbd79, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.850189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.864046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 795/1574] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:39 2022] PUT /identity/v3/projects/977038321e5a41e8ab137fede4d118d7/users/444719e34bf2457b9a58e182ef7fbd79/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:39.865072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.896447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0176f0a-089a-45fa-8406-7c72c392ed44 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.925307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:39.931719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a73eb-1a3b-4b25-8fed-d68594e7ee1f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:39.932992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:39.939339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.939990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.940405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.940679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.941315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.943247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.958872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:39.959556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:39.959835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:39.960107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:39.960775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:39.962752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:39.974174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 796/1575] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.974174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:39.991033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 780/1576] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 393 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:39.991871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.045930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f29dd2-2645-455a-938b-510f005d701a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.051350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.052043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.052355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.052624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.053268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.054411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ac75-476f-4a1a-a062-8ac0059ebdbb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.054926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.059083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.059836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.060135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.060447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.061035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.062943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.068705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 797/1577] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.069460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.081609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b22e1a-180d-4cec-b141-da650696b504 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.086468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.087066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.087351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.087786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.088421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.090336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.095496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 781/1578] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:39 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.096370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.132100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 798/1579] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.132100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.162197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a56b-95ee-455e-93b8-ffab18b39b22 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.192672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c483-1af7-45a0-b229-db37e7dd6d21 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.197505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38b56f-5944-43f8-9fab-1877b3c08733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.198166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38b56f-5944-43f8-9fab-1877b3c08733 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.198507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38b56f-5944-43f8-9fab-1877b3c08733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.198838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38b56f-5944-43f8-9fab-1877b3c08733 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.199564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38b56f-5944-43f8-9fab-1877b3c08733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.201506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38b56f-5944-43f8-9fab-1877b3c08733 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.212018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:40.221762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.222445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.222735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.223276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.224240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.228690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 799/1580] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.228859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.228969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.228969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.228969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.236240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 782/1581] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.237093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.246722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba515b-51e8-4986-a7e0-9ddd3bc6b3a4 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.247477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:40.248956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:40.251367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.252006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.252281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.252552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.253177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.255085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.256796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 800/1582] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:40.260005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.267524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8d22-e324-4023-ac7d-6e735f4a398e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.272942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.273558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.273894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.274180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.274858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.276856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.279208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 783/1583] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.279860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.292184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940f901-78c6-496d-b0d0-22b0d61ab8bb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.297333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.298043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.298329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.298597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] PATH_INFO: `/v3/projects/808b55bc616349cfb31ee041ce69c976/users/be2ad8e7eefe447ca1374d49696b1318/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.303782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 801/1584] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.304593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.314519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808b55bc616349cfb31ee041ce69c976, user_id=be2ad8e7eefe447ca1374d49696b1318, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.315352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a9cbc-c39a-4cfd-a723-64a83bbddede admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.315597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:44:40.315597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:44:40.315597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:44:40.317555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.320044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.320582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.320864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.321131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.321953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.324082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.331170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 784/1585] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:40 2022] PUT /identity/v3/projects/808b55bc616349cfb31ee041ce69c976/users/be2ad8e7eefe447ca1374d49696b1318/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:40.332206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.350825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 802/1586] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.351684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.356568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:40.362696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecd70e-3d2c-4ff6-b005-9b000583472b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.399968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edddfd-1cb4-4c86-a0c7-dd8d84d94ac2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.404675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.405280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.405556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.405825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.406446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.408605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.424591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.425316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.425607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.425893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.426536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.428175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.431401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 785/1587] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:40.432321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.440590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 803/1588] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.441276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.445886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e572bc3-848f-49d5-83bf-f448f5bb99a6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.451472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.452164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.452525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.452781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.453433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.455056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.465321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 786/1589] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.466260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.476565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b55a3-b649-4a06-b75b-71eed3e9aa09 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.481264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.481857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.482133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.482427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.483062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.484692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.496103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 787/1590] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.497042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.502547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a16e5-d436-4628-a365-03b8ff9f7654 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.506190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.506603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e71c9-95dc-46e2-b025-2e1bab62cdcc admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.506795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.507085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.507334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.511171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09bd6d-21b6-4050-8730-be71f4b4b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.511775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09bd6d-21b6-4050-8730-be71f4b4b8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.512051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09bd6d-21b6-4050-8730-be71f4b4b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.512316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09bd6d-21b6-4050-8730-be71f4b4b8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.512939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09bd6d-21b6-4050-8730-be71f4b4b8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.514525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09bd6d-21b6-4050-8730-be71f4b4b8f4 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.516559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.518183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.524364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 788/1591] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.525599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.525885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.526155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.529107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 804/1592] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:40.529907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.541361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9c73b-9ced-4e61-97cc-df6ab3581c01 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.547046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.547839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.549317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.549574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:40.549909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.550784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:40.551111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.553193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.563092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 789/1593] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:40.563092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.565769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 805/1594] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.568770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.579276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb706258-ed65-412d-bb48-15ee9e2399a7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.579541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f53912-55a7-4644-a78c-441fe341e5d3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.584439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.584612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.585078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.585197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.585376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.585488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.585649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.585752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.586265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.586380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.587927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.588034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.600333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 790/1595] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.601057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 806/1596] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.601949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.602212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.614121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397ca1-33ed-402a-90dc-bb2dd6e10c24 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.615700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e275da-0c99-4660-aef2-8ffd0440f281 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.619826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.620638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.620773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.621047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.621334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] PATH_INFO: `/v3/projects/0900418261fc4370bbf30ffcc668c4c5/users/89b20ec0ede5470eaa33e1ea934991d9/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.621526 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.621905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.622563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.623013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.624658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.636880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 807/1597] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.638647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.641752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0900418261fc4370bbf30ffcc668c4c5, user_id=89b20ec0ede5470eaa33e1ea934991d9, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.644773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.651708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4934d-3731-44c3-ab54-967a0a3f8319 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.657987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.658744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.659082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.659493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.660289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 791/1598] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:40 2022] PUT /identity/v3/projects/0900418261fc4370bbf30ffcc668c4c5/users/89b20ec0ede5470eaa33e1ea934991d9/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:40.660908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.663639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.665888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.680966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 808/1599] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.685147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.689314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:40.692654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:40.702887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd296-d93e-4627-9cba-f5455687c763 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.739872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8f69b-187b-4289-ab8c-ae01e2d5270f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.746337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.746337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.746337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.746337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.747136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.753522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.763777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.764396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.764698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.764965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.768595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 792/1600] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.769535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.775302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.776992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.783325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2081ca-16fb-40d6-988a-4d69e69d663a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.784077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:40.790611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.791236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 809/1601] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:40.791454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.791736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.792006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.792649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.792984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.794390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.807494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 793/1602] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:40.808630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.823339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5018252-c829-4450-bfa2-bf4d4b8b8fd7 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.824289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bedd35-8bfa-4748-9e24-e8858d3ad307 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.830583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef2057-a77f-4bd0-a820-e4cfde65656d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.831269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef2057-a77f-4bd0-a820-e4cfde65656d admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.831716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef2057-a77f-4bd0-a820-e4cfde65656d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.831848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.832047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef2057-a77f-4bd0-a820-e4cfde65656d admin admin] PATH_INFO: `/v3/projects/e95a6395c5da40eea6867858df27003b/users/6a569f7f946940d28f220d1485099e4c/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.832572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.833146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.833265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] PATH_INFO: `/v3/projects/3417cf79d7d748d483e8aeb8289b75f6/users/6974a4c8fffd4bc2933e754770cb912e/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.853087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef2057-a77f-4bd0-a820-e4cfde65656d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95a6395c5da40eea6867858df27003b, user_id=6a569f7f946940d28f220d1485099e4c, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.856041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef2057-a77f-4bd0-a820-e4cfde65656d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.857098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3417cf79d7d748d483e8aeb8289b75f6, user_id=6974a4c8fffd4bc2933e754770cb912e, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:40.859871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:40.876654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 794/1603] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:40 2022] PUT /identity/v3/projects/e95a6395c5da40eea6867858df27003b/users/6a569f7f946940d28f220d1485099e4c/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:40.878466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.879218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.879942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:40.889629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 810/1604] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:40 2022] PUT /identity/v3/projects/3417cf79d7d748d483e8aeb8289b75f6/users/6974a4c8fffd4bc2933e754770cb912e/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:40.890615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:40.908499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:40.910360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:40.987847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3033547-c72e-4683-90a7-396db3eb0f66 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:40.994817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:40.995726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:40.996069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:40.996402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] PATH_INFO: `/v3/projects/0d671773b58240b1988f7c56ec6753b5/users/f417a94fed274bfbb88d23677a4877a7/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.001750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 795/1605] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:41.003607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.015723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b72634-62fe-4f1f-a61e-7ec5c2481955 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.024127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d671773b58240b1988f7c56ec6753b5, user_id=f417a94fed274bfbb88d23677a4877a7, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.026505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.037123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bda09-1c04-47c6-8a0b-a6cfd8c333e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.040042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bda09-1c04-47c6-8a0b-a6cfd8c333e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.041497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bda09-1c04-47c6-8a0b-a6cfd8c333e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.042753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bda09-1c04-47c6-8a0b-a6cfd8c333e0 admin admin] PATH_INFO: `/v3/projects/ec8b2a41082e4947867cb5470a6cb87a/users/f86526a0f95941428edbb436fd571ff4/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.070220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 811/1606] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:40 2022] PUT /identity/v3/projects/0d671773b58240b1988f7c56ec6753b5/users/f417a94fed274bfbb88d23677a4877a7/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:41.081069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.102040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bda09-1c04-47c6-8a0b-a6cfd8c333e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8b2a41082e4947867cb5470a6cb87a, user_id=f86526a0f95941428edbb436fd571ff4, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.111334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bda09-1c04-47c6-8a0b-a6cfd8c333e0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.143468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 796/1607] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:41 2022] PUT /identity/v3/projects/ec8b2a41082e4947867cb5470a6cb87a/users/f86526a0f95941428edbb436fd571ff4/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:41.145503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50223d-5c0e-45d3-af46-15297d422aa2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.149349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50223d-5c0e-45d3-af46-15297d422aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.149923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50223d-5c0e-45d3-af46-15297d422aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.190372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8a50223d-5c0e-45d3-af46-15297d422aa2 None None] MFA Rules not processed for user `6a569f7f946940d28f220d1485099e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:41.191883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8a50223d-5c0e-45d3-af46-15297d422aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:41.218976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289b9de-a5ca-4a2e-a6c9-2b72aeede187 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.262008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:41.300862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 797/1608] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:41.302129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.302934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44819029-0084-489f-bdab-4a84a745172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.303415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.303731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.303955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44819029-0084-489f-bdab-4a84a745172a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.304332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44819029-0084-489f-bdab-4a84a745172a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.304727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44819029-0084-489f-bdab-4a84a745172a admin admin] PATH_INFO: `/v3/projects/509435dd290c472a911792a21b32534f/users/92e077689b0f41cb878f630a93313b2b/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.320120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:41.324835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44819029-0084-489f-bdab-4a84a745172a admin admin] RBAC: Authorizing `identity:create_grant(project_id=509435dd290c472a911792a21b32534f, user_id=92e077689b0f41cb878f630a93313b2b, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.327676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44819029-0084-489f-bdab-4a84a745172a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.342822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:41.344332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:41.356206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 798/1609] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:41.359933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.369362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 812/1610] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:41 2022] PUT /identity/v3/projects/509435dd290c472a911792a21b32534f/users/92e077689b0f41cb878f630a93313b2b/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:41.370650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aee34-2566-4beb-b9eb-bd21b9456b0a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.371005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aee34-2566-4beb-b9eb-bd21b9456b0a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.371348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aee34-2566-4beb-b9eb-bd21b9456b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.393389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0f6aee34-2566-4beb-b9eb-bd21b9456b0a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:41.394624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0f6aee34-2566-4beb-b9eb-bd21b9456b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:41.491626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64266a3-ecf7-4c52-9c82-2c3fa0e73145 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.495784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 813/1611] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:41.497792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d632b-80db-4fd7-9002-7493b3593ebd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.497792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d632b-80db-4fd7-9002-7493b3593ebd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.498066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d632b-80db-4fd7-9002-7493b3593ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.556378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-804d632b-80db-4fd7-9002-7493b3593ebd None None] MFA Rules not processed for user `f86526a0f95941428edbb436fd571ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:41.558274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-804d632b-80db-4fd7-9002-7493b3593ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:41.559339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.560297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.560747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.561309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.574032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.577248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.589542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 799/1612] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:41.590387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.620705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81366b95-bcda-4dd5-8fe6-945424823138 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.627365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.628262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.629075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.629497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.630396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.633631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.647359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 814/1613] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:41.647359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.647359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.647359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.692603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 800/1614] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:41.693676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.696074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] MFA Rules not processed for user `92e077689b0f41cb878f630a93313b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:41.699786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:41.773135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f775-1116-48d9-899d-9c63b8ce5c01 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.777855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.778810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.779207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 815/1615] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:41.779704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.780199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.780564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.795158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767faa5a-a8e0-44f5-9f96-230ab28a5204 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.800533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a7975-55ba-4eb2-a61e-91930373cf55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.801278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7975-55ba-4eb2-a61e-91930373cf55 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.801680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7975-55ba-4eb2-a61e-91930373cf55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.801983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7975-55ba-4eb2-a61e-91930373cf55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.808512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:41.836148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:41.882748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a7975-55ba-4eb2-a61e-91930373cf55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.884441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a7975-55ba-4eb2-a61e-91930373cf55 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.890975 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.893488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.896740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 816/1616] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:41.900213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc5719-a60e-48de-9d5e-5e742f11bba6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.900213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc5719-a60e-48de-9d5e-5e742f11bba6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.900213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc5719-a60e-48de-9d5e-5e742f11bba6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.904686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 817/1617] 10.222.0.55 () {54 vars in 961 bytes} [Sat Sep 24 19:44:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:44:41.905632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc5719-a60e-48de-9d5e-5e742f11bba6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.927261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc5719-a60e-48de-9d5e-5e742f11bba6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.933717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 801/1618] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:41.935374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:41.937821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18bda2-deab-478d-88a1-aa5762838368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.938754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18bda2-deab-478d-88a1-aa5762838368 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.941623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18bda2-deab-478d-88a1-aa5762838368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.942075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18bda2-deab-478d-88a1-aa5762838368 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.942747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18bda2-deab-478d-88a1-aa5762838368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.950163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18bda2-deab-478d-88a1-aa5762838368 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.951791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d058d3-7cd0-4f03-844d-deec25d795aa service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:41.957406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:41.958232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:41.958511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:41.958786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:41.959444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:41.961838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:41.999890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 818/1619] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:42.003980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.004626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.005381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.009600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 802/1620] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:42.013048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.106533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:42.108521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:42.196034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1220a9-c4c7-481b-b42d-d9787035e071 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.209080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.209788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.210075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.210355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.211008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 819/1621] 10.222.0.55 () {58 vars in 1058 bytes} [Sat Sep 24 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:42.211149 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.212840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.222127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.234637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 803/1622] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:42.238222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280473bb-fb2f-4d6f-8404-b63e5d63cec8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.239418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.245157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.245677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.246113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.246477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.247265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.249829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.273132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a90e-c6e6-4e81-bd40-a5ec94f165fe admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.283463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.285040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.285421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.286576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.302248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 820/1623] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:42 2022] POST /identity/v3/users => generated 308 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:42.303825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.331072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b2f57-ebb7-4a7c-abd5-45f4871a17e7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.348351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.350884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.351287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.352713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] PATH_INFO: `/v3/projects/39f4c19546404d0db037946415f8810e/users/191443d4f08b4ed5bc46cd5f8d1f7b49/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.395401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f4c19546404d0db037946415f8810e, user_id=191443d4f08b4ed5bc46cd5f8d1f7b49, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.413058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.438909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 821/1624] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:42 2022] PUT /identity/v3/projects/39f4c19546404d0db037946415f8810e/users/191443d4f08b4ed5bc46cd5f8d1f7b49/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:42.444295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.457469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.459107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.481880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 804/1625] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:42.495589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.513503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:42.528361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43033196-b9fb-4c45-a0b3-d8690be70b1a service nova] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.634706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8ec25-efce-4d17-b9f0-ab50583375cf admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.637741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:42.648245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.648935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.649239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.649491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.659953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.661701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.680266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.680892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.681234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.681655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.683513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.685907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.691931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 822/1626] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:42.695806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.706466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96e97-9330-4ad6-87c8-ca70f8b31b2e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.711846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.712575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.712907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.713286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] PATH_INFO: `/v3/projects/80f20ab754264233be8cc6153f8914e7/users/118d63b851054b29b2aa6f98bd2614ea/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.721186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 805/1627] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:42.722171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.741128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f20ab754264233be8cc6153f8914e7, user_id=118d63b851054b29b2aa6f98bd2614ea, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.744523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.744684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429bca-7519-4516-b2df-d132aca0c712 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:42.762850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1a8ff-e2b1-4893-ba5a-d62f768ef07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:42.763570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a8ff-e2b1-4893-ba5a-d62f768ef07f admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.763854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a8ff-e2b1-4893-ba5a-d62f768ef07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.764124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a8ff-e2b1-4893-ba5a-d62f768ef07f admin admin] PATH_INFO: `/v3/projects/39f4c19546404d0db037946415f8810e/users/191443d4f08b4ed5bc46cd5f8d1f7b49/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.766097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1a8ff-e2b1-4893-ba5a-d62f768ef07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f4c19546404d0db037946415f8810e, user_id=191443d4f08b4ed5bc46cd5f8d1f7b49, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:42.769481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1a8ff-e2b1-4893-ba5a-d62f768ef07f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:42.769657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 823/1628] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:42 2022] PUT /identity/v3/projects/80f20ab754264233be8cc6153f8914e7/users/118d63b851054b29b2aa6f98bd2614ea/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:44:42.773819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:42.797316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 806/1629] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:42 2022] PUT /identity/v3/projects/39f4c19546404d0db037946415f8810e/users/191443d4f08b4ed5bc46cd5f8d1f7b49/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:42.810844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:42.811214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:42.811516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:42.877209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] MFA Rules not processed for user `191443d4f08b4ed5bc46cd5f8d1f7b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:42.880407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:42.931618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07f10d-dbbe-445a-bfd8-c9a9fa4de054 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:43.026148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 807/1630] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:43.034578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:43.101209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:43.102087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.102515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.102962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.104862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:43.106643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:43.124647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 824/1631] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:43.126619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:43.203724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71569e-59fe-461c-9d20-d9dba862824b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:43.203724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da2f6-8437-4bf8-aa45-0b44bb4cd554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:43.203724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da2f6-8437-4bf8-aa45-0b44bb4cd554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.203724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da2f6-8437-4bf8-aa45-0b44bb4cd554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.205270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da2f6-8437-4bf8-aa45-0b44bb4cd554 admin admin] PATH_INFO: `/v3/projects/80f20ab754264233be8cc6153f8914e7/users/118d63b851054b29b2aa6f98bd2614ea/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.205270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54da2f6-8437-4bf8-aa45-0b44bb4cd554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f20ab754264233be8cc6153f8914e7, user_id=118d63b851054b29b2aa6f98bd2614ea, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:43.205270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54da2f6-8437-4bf8-aa45-0b44bb4cd554 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:43.205270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 825/1632] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:43 2022] PUT /identity/v3/projects/80f20ab754264233be8cc6153f8914e7/users/118d63b851054b29b2aa6f98bd2614ea/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:43.205270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.205934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.205934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.224231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f121c-bd6e-423a-95f3-0a61c2ad63f0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:43.255768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] MFA Rules not processed for user `118d63b851054b29b2aa6f98bd2614ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:43.257341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:43.298547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:43.301983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:43.302883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.303354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.303821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.346547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:43.438164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:43.442693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:43.469363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 808/1633] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:44:43.486705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 826/1634] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:43 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:43.497135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:43.511780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba033d-ded7-4e1c-9cff-8cc464f37727 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:43.523670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e74c3-cdb2-46dd-83b6-5336218a6350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:43.524538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e74c3-cdb2-46dd-83b6-5336218a6350 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.524809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e74c3-cdb2-46dd-83b6-5336218a6350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.525086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e74c3-cdb2-46dd-83b6-5336218a6350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.544720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:43.551638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e74c3-cdb2-46dd-83b6-5336218a6350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:43.553420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e74c3-cdb2-46dd-83b6-5336218a6350 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:43.562202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45053014-ef32-442b-9076-c7a00ba0f460 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:43.568080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:43.569095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.569542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.569967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.574867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 809/1635] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:43.647934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261de29-985a-485d-b0ad-bfa6f9e539bc None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.648252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261de29-985a-485d-b0ad-bfa6f9e539bc None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.648953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261de29-985a-485d-b0ad-bfa6f9e539bc None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.650443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 810/1636] 10.222.0.55 () {54 vars in 961 bytes} [Sat Sep 24 19:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:44:43.663021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94fec3-fccc-4acd-a48d-f673cbaa65fb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.663404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94fec3-fccc-4acd-a48d-f673cbaa65fb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.663780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94fec3-fccc-4acd-a48d-f673cbaa65fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.701049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1e94fec3-fccc-4acd-a48d-f673cbaa65fb None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:43.706757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1e94fec3-fccc-4acd-a48d-f673cbaa65fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:43.778323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:43.784658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:43.809061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 827/1637] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:43.921911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:43.935955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 811/1638] 10.222.0.55 () {58 vars in 1058 bytes} [Sat Sep 24 19:44:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:43.942827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98558e71-3e01-4235-aa2d-54f0dd3a06d6 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.943109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98558e71-3e01-4235-aa2d-54f0dd3a06d6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.943387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98558e71-3e01-4235-aa2d-54f0dd3a06d6 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.948491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 812/1639] 10.222.0.55 () {54 vars in 912 bytes} [Sat Sep 24 19:44:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:44:43.950677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:43.950969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:43.951245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:43.993902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:43.995211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:44.021042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 813/1640] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:44:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:44.034630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:44.046984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74609b51-ff4f-4b5c-a284-b0159f0067f0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:44.064324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c18e-e0c5-4ae4-acf8-22a1abf7b329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:44.072869 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c18e-e0c5-4ae4-acf8-22a1abf7b329 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.073559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c18e-e0c5-4ae4-acf8-22a1abf7b329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.074081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c18e-e0c5-4ae4-acf8-22a1abf7b329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.123336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493c18e-e0c5-4ae4-acf8-22a1abf7b329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:44.124860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493c18e-e0c5-4ae4-acf8-22a1abf7b329 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:44.136120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439a882-1ef1-4758-88e0-c0adb4437e08 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:44.155993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:44.156666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.156949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.157791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] PATH_INFO: `/v3/users/92e077689b0f41cb878f630a93313b2b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.164396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e077689b0f41cb878f630a93313b2b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:44.166845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:44.171220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 814/1641] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:44:44.319655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e077689b0f41cb878f630a93313b2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:44.330110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:44.396328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e077689b0f41cb878f630a93313b2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:44.422399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e077689b0f41cb878f630a93313b2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:44.423913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e077689b0f41cb878f630a93313b2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:44.451357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-39d8ae80-2271-401c-8c7c-de5aea840749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e077689b0f41cb878f630a93313b2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:44.473067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb845-9900-4679-b391-75e9bd41a2c5 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.473639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb845-9900-4679-b391-75e9bd41a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.474576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb845-9900-4679-b391-75e9bd41a2c5 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.476957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 828/1643] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:44:43 2022] DELETE /identity/v3/users/92e077689b0f41cb878f630a93313b2b => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:44.479964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 815/1643] 10.222.0.55 () {54 vars in 912 bytes} [Sat Sep 24 19:44:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:44:44.493180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.494454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.494921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.495365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.495877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.496664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.533591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:44.535171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:44.538013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:44.539307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:44.720156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 816/1644] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:44.721549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:44.723622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 829/1645] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:44.728217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:44.753577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:44.807976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:44.810792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:44.944862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ee3ad-02ca-49ef-aa82-d5ad940393c7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:44.956670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80367090-602b-445c-a268-3e08b1559d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:44.957353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80367090-602b-445c-a268-3e08b1559d6a service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.957639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80367090-602b-445c-a268-3e08b1559d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.958484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80367090-602b-445c-a268-3e08b1559d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.972838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef4dc-9a11-43f2-91c7-2e498b81bf02 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:44.977744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:44.978379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:44.978810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:44.978904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:44.990180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80367090-602b-445c-a268-3e08b1559d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:44.993696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80367090-602b-445c-a268-3e08b1559d6a service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:44.998401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:44.999791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:45.023612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 830/1646] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:44:45.030936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 817/1647] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:45.146618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:45.148052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:45.164588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50046-152e-45de-8564-4eafef7d72a7 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:45.172166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:45.173299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:45.173828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:45.174269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] PATH_INFO: `/v3/users/f86526a0f95941428edbb436fd571ff4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:45.176167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86526a0f95941428edbb436fd571ff4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:45.178799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:45.268096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86526a0f95941428edbb436fd571ff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:45.308139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86526a0f95941428edbb436fd571ff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:45.311676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:45.326874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86526a0f95941428edbb436fd571ff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:45.327967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86526a0f95941428edbb436fd571ff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:45.345096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86526a0f95941428edbb436fd571ff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:45.356714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 831/1648] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:44:45 2022] DELETE /identity/v3/users/f86526a0f95941428edbb436fd571ff4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:45.358063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:45.358543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:45.358625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:45.398519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:45.408318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:45.520300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 818/1649] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:45.544774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:45.546508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:45.646194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea11691-e5d1-47d9-bd01-73c449106f69 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:45.650931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:45.653992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:45.653992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:45.653992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:45.669122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:45.671125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:45.687723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 832/1650] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:47.327767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:47.332655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:47.393460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438bcd-c53b-4347-bffd-17762583fd2a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:47.406259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:47.407289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:47.408215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:47.408688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] PATH_INFO: `/v3/projects/509435dd290c472a911792a21b32534f` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:47.410481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509435dd290c472a911792a21b32534f)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:47.413543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:47.493185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:47.494007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509435dd290c472a911792a21b32534f', 'request_id': 'req-d1392af8-d1c6-4a94-9cfc-dabe790914f3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.495268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:47.508096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509435dd290c472a911792a21b32534f', 'request_id': 'req-d1392af8-d1c6-4a94-9cfc-dabe790914f3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.516876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e10f6-b22c-4640-8c53-0468f1bfc010 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:47.518271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509435dd290c472a911792a21b32534f', 'request_id': 'req-d1392af8-d1c6-4a94-9cfc-dabe790914f3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.536828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] The token cache is being invalidate because project 509435dd290c472a911792a21b32534f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:44:47.537934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.539356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.542514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 819/1651] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:44:47 2022] DELETE /identity/v3/projects/509435dd290c472a911792a21b32534f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:47.547054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:47.550486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:47.650983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:47.695535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:47.696400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:47.696633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:47.697132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] PATH_INFO: `/v3/projects/ec8b2a41082e4947867cb5470a6cb87a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:47.698680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8b2a41082e4947867cb5470a6cb87a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:47.700504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:44:47.700504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Sep 24 19:44:47.700504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: Sep 24 19:44:47.706986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:47.711565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392af8-d1c6-4a94-9cfc-dabe790914f3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:47.719202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:47.720164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:47.720380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:47.720596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] PATH_INFO: `/v3/users/118d63b851054b29b2aa6f98bd2614ea` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:47.721903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=118d63b851054b29b2aa6f98bd2614ea)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:47.723058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: WARNING py.warnings [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 19:44:47.723058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: warnings.warn(msg) Sep 24 19:44:47.723058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: Sep 24 19:44:47.726161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:47.822280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118d63b851054b29b2aa6f98bd2614ea'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.834089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118d63b851054b29b2aa6f98bd2614ea'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.835884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118d63b851054b29b2aa6f98bd2614ea'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.836825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118d63b851054b29b2aa6f98bd2614ea'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.857976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8b2a41082e4947867cb5470a6cb87a', 'request_id': 'req-a94783cf-c258-4777-9b5c-8559fe6f870c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.878862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8b2a41082e4947867cb5470a6cb87a', 'request_id': 'req-a94783cf-c258-4777-9b5c-8559fe6f870c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.882429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d59f6b2a-49b7-4db6-9d84-5b601829a300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118d63b851054b29b2aa6f98bd2614ea'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.893819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8b2a41082e4947867cb5470a6cb87a', 'request_id': 'req-a94783cf-c258-4777-9b5c-8559fe6f870c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.929072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] The token cache is being invalidate because project ec8b2a41082e4947867cb5470a6cb87a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:44:47.929388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.930203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a94783cf-c258-4777-9b5c-8559fe6f870c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:47.932507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 833/1652] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:44:47 2022] DELETE /identity/v3/projects/ec8b2a41082e4947867cb5470a6cb87a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:44:47.937888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:47.938687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:47.938883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:47.963322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 820/1653] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:44:47 2022] DELETE /identity/v3/users/118d63b851054b29b2aa6f98bd2614ea => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:47.985068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:48.005180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:48.044632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:48.167025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 834/1654] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:48.176925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.177216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.177491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.217181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:48.220459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:48.269570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 821/1655] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:48.270869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:48.298226 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0212996-d4ad-4e5d-a5d0-8466bb9dd3e6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:48.319092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:48.319755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.323595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.323982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.325787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:48.329537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:48.335390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:48.370719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:48.371351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 835/1656] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:48.387001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd8af-7975-4584-92c5-fcf9e08f9ae8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:48.399363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:48.400053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.400401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.400588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.401236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:48.403187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:48.466228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 822/1657] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:48 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:48.468663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:48.504844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb6fd4-3143-4211-9773-1bb3bdfc7aec admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:48.681863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:48.684973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.684973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.684973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.684973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:48.689244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:48.724647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 836/1658] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:48 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:48.728917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:48.744141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cebeca-6064-4b30-9ff3-1b8df40b250d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:48.761639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:48.762660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.763228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.763785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.764795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:48.768063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:48.805749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 823/1659] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:48.807111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:48.825340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f153aa-ac39-4608-893a-1e5bae6e9735 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:48.844134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e883ab3-ae71-442f-95d0-41240aa97a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:48.845982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883ab3-ae71-442f-95d0-41240aa97a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.847698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883ab3-ae71-442f-95d0-41240aa97a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.848135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883ab3-ae71-442f-95d0-41240aa97a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.848960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e883ab3-ae71-442f-95d0-41240aa97a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:48.850977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e883ab3-ae71-442f-95d0-41240aa97a2d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:48.883509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 837/1660] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:48.890681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:48.916973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19da75-9b27-427b-a877-6b7428e77cc1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:48.924674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:48.925410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:48.925914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:48.926264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] PATH_INFO: `/v3/projects/e809723e7474499fb0e097d3d4f4e117/users/eb5035771789445990e1f672774b4d2f/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:48.955846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e809723e7474499fb0e097d3d4f4e117, user_id=eb5035771789445990e1f672774b4d2f, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:48.959033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:48.997801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 824/1661] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:48 2022] PUT /identity/v3/projects/e809723e7474499fb0e097d3d4f4e117/users/eb5035771789445990e1f672774b4d2f/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:49.013529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:49.013988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:49.014271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:49.142944 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] MFA Rules not processed for user `eb5035771789445990e1f672774b4d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:49.145484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:49.343379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 838/1662] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:49.347641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:49.533964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:49.536286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:49.540365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:49.563753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2463d-0e80-4e1d-9a08-42227b39fa3d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:49.581475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:49.590804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:49.591478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:49.591757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:49.592027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:49.592677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:49.594703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:49.655213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 825/1663] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:49 2022] POST /identity/v3/projects => generated 375 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:49.656359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:49.700851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a616374-fd06-402a-a77b-83120f28d4a7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:49.809882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15eb40-90be-490e-ae48-8a0e69edbc25 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:49.828446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:49.829161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:49.829601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:49.830069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:49.830723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:49.836834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:49.841394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:49.842118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:49.842357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:49.842734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] PATH_INFO: `/v3/users/6a569f7f946940d28f220d1485099e4c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:49.843957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a569f7f946940d28f220d1485099e4c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:49.846746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:49.876265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 826/1664] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:49 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:44:49.878729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:49.883875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:49.911525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00435fdc-43bf-4688-ac7d-6a33e5d8a9a5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:49.926198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:49.927770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:49.928002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:49.928270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] PATH_INFO: `/v3/projects/80f20ab754264233be8cc6153f8914e7` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:49.932710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f20ab754264233be8cc6153f8914e7)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:49.934450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a569f7f946940d28f220d1485099e4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:49.937059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:49.968211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a569f7f946940d28f220d1485099e4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:49.994763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a569f7f946940d28f220d1485099e4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:49.996703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a569f7f946940d28f220d1485099e4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.020045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a569f7f946940d28f220d1485099e4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.048962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 839/1665] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:44:49 2022] DELETE /identity/v3/users/6a569f7f946940d28f220d1485099e4c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:50.053148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.073833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:50.129621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f20ab754264233be8cc6153f8914e7', 'request_id': 'req-70da4e4a-ad09-4f3d-a769-59329141de29'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.150384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f20ab754264233be8cc6153f8914e7', 'request_id': 'req-70da4e4a-ad09-4f3d-a769-59329141de29'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.162408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f20ab754264233be8cc6153f8914e7', 'request_id': 'req-70da4e4a-ad09-4f3d-a769-59329141de29'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.189944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] The token cache is being invalidate because project 80f20ab754264233be8cc6153f8914e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:44:50.190248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.192010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-70da4e4a-ad09-4f3d-a769-59329141de29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:50.193625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 827/1666] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:44:49 2022] DELETE /identity/v3/projects/80f20ab754264233be8cc6153f8914e7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:50.194875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0f49d-ef69-4a7f-a4c6-6ed2b105320c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.195157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0f49d-ef69-4a7f-a4c6-6ed2b105320c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.195544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0f49d-ef69-4a7f-a4c6-6ed2b105320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.235526 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:50.243126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-64d0f49d-ef69-4a7f-a4c6-6ed2b105320c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:50.244754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-64d0f49d-ef69-4a7f-a4c6-6ed2b105320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:50.307717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7aa2-d623-4706-92d5-3d535c4816fe admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.322226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:50.377739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 828/1667] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:50.380898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0feb5-f953-436b-ac75-19385165259e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.381186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0feb5-f953-436b-ac75-19385165259e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.381618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0feb5-f953-436b-ac75-19385165259e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.403978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-46d0feb5-f953-436b-ac75-19385165259e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:50.405361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-46d0feb5-f953-436b-ac75-19385165259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:50.409379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.410272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.410761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.411253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.412265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.414704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.418190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 829/1668] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:50.428235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.428853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.429304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.437258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 840/1669] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:50.442363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.445003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:50.457253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:50.458691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:50.470756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 830/1670] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:50.481374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671f45-e183-47a3-b78f-555d69170745 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.484765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.487815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.490357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.490791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.491860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] PATH_INFO: `/v3/projects/32002cc4096e4071ad3e61193ad31e02/users/e0b0da2448684bf78ec14e502adca840/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.498737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6a14-4980-4c83-9825-c811e03f7e69 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.511879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.512419 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.512700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.512971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.513636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:50.513739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32002cc4096e4071ad3e61193ad31e02, user_id=e0b0da2448684bf78ec14e502adca840, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.513939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.515630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.516973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.535994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 841/1671] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:50 2022] PUT /identity/v3/projects/32002cc4096e4071ad3e61193ad31e02/users/e0b0da2448684bf78ec14e502adca840/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:50.540843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.554318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 831/1672] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:50.558510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.570414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057746d-f545-4518-baad-7c812fdec64b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.670904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0305e0e-c8e2-4145-9d55-5658704199b1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.675610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.676728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.676883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.677310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.678115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.678472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.679211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.679520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.680111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.680746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.680890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.683217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.702497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 842/1673] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:50.702497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.702497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:50.726321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 832/1674] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:50 2022] POST /identity/v3/projects => generated 393 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:44:50.727326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.805916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847051-2a97-4144-9ee5-8bc150b2be67 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.814329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:50.889128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fc43e-5051-4810-88d1-ad178277bc49 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:50.902257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f681a-6884-44c3-97b3-b4230db532aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.902900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f681a-6884-44c3-97b3-b4230db532aa service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.903605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f681a-6884-44c3-97b3-b4230db532aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.903897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f681a-6884-44c3-97b3-b4230db532aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.904001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:50.904693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.904972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.905247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] PATH_INFO: `/v3/projects/32002cc4096e4071ad3e61193ad31e02/users/e0b0da2448684bf78ec14e502adca840/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:50.914458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=32002cc4096e4071ad3e61193ad31e02, user_id=e0b0da2448684bf78ec14e502adca840, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.917557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f681a-6884-44c3-97b3-b4230db532aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:50.918154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.919959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f681a-6884-44c3-97b3-b4230db532aa service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:50.950213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 833/1675] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:50 2022] PUT /identity/v3/projects/32002cc4096e4071ad3e61193ad31e02/users/e0b0da2448684bf78ec14e502adca840/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:50.950401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 843/1676] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:50.950475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:50.952780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:50.953088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:50.953427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.000028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] MFA Rules not processed for user `e0b0da2448684bf78ec14e502adca840`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:51.004084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:51.046073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:51.075730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f811e1-ee9e-49da-8ddc-69f2aa20d2fe admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:51.082682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:51.083460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.083752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.084366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.085187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:51.087285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:51.112996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 834/1677] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:50 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:51.117684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:51.135236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 844/1678] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:51.135457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637147e-81b9-40b7-b70d-e558b468cae8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:51.141286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:51.142113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.145184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.145552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.146309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:51.150066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:51.150337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:51.163853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:51.172797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 835/1679] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:51.180390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:51.205721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca3473-cd6f-4059-9972-28e4431816ee admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:51.223273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:51.223997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.224478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.224747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.225407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:51.227050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:51.254374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 836/1680] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:51.258602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:51.273239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f57d-d9a6-4bb9-adae-03727c51302a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:51.292879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814ff1-f1c8-4b8e-a2af-781eb79409ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:51.293367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814ff1-f1c8-4b8e-a2af-781eb79409ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.293649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814ff1-f1c8-4b8e-a2af-781eb79409ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.294062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814ff1-f1c8-4b8e-a2af-781eb79409ee admin admin] PATH_INFO: `/v3/projects/be93e774048a472abdc3ce1bf16e74a1/users/4cd5ac8eedef455c9470e4c6580d70ef/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.314396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f1b7a-2cd9-4b7d-aa7f-267df1c0e76e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:51.322024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:51.334213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.334950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.335508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.336615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814ff1-f1c8-4b8e-a2af-781eb79409ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=be93e774048a472abdc3ce1bf16e74a1, user_id=4cd5ac8eedef455c9470e4c6580d70ef, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:51.338697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:51.339355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814ff1-f1c8-4b8e-a2af-781eb79409ee admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:51.369773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 837/1681] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:51 2022] PUT /identity/v3/projects/be93e774048a472abdc3ce1bf16e74a1/users/4cd5ac8eedef455c9470e4c6580d70ef/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:44:51.381310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.381595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.381865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.457118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] MFA Rules not processed for user `4cd5ac8eedef455c9470e4c6580d70ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:51.458353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:51.518371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:51.542587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:51.547083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:51.585808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 845/1682] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:51.631842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:51.632294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 838/1683] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:44:51.660819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84406b0-5f15-4845-80d9-989448288ff6 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:51.837386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:51.837943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:51.840600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:51.840882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:51.841577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:51.848777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:51.905620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 846/1684] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:51 2022] POST /identity/v3/projects => generated 393 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:44:51.910163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:51.928793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031c413-ab19-447a-9027-7ec9753ec96f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:52.064484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:52.065208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:52.065552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:52.065896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:52.067082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:52.068669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:52.116063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 839/1685] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:51 2022] POST /identity/v3/users => generated 303 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:52.122203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:52.165283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb82f51-c984-4820-870e-5c3d1292cd57 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:52.184362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:52.195017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:52.196458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:52.197493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:52.198399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:52.237196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:52.281146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 847/1686] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:52.286171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:52.294792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:52.302311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0258-8c09-4cd6-99ff-0c9dd8651a42 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:52.315831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:52.317487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:52.317487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:52.317487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] PATH_INFO: `/v3/projects/84b8364a75cb44ac90ede2f5891c2e15/users/e126dd20c41946e3ad8ffa0a90fa59e8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:52.359316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b8364a75cb44ac90ede2f5891c2e15, user_id=e126dd20c41946e3ad8ffa0a90fa59e8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:52.366500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:52.401997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 840/1687] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:52 2022] PUT /identity/v3/projects/84b8364a75cb44ac90ede2f5891c2e15/users/e126dd20c41946e3ad8ffa0a90fa59e8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:52.405084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:52.437322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75239f27-f42f-4cad-9018-dfc8ca326fff admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:52.688881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d923c6-3439-4bc1-9c15-77a62c10de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:52.690236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d923c6-3439-4bc1-9c15-77a62c10de22 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:52.690654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d923c6-3439-4bc1-9c15-77a62c10de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:52.691061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d923c6-3439-4bc1-9c15-77a62c10de22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:52.691981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d923c6-3439-4bc1-9c15-77a62c10de22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:52.694473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d923c6-3439-4bc1-9c15-77a62c10de22 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:52.714796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:52.724528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 848/1688] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:52.728726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07102f3-ec87-4b39-a97d-c6fa38885d28 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:52.736212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:52.738020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:52.738020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:52.738020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] PATH_INFO: `/v3/projects/84b8364a75cb44ac90ede2f5891c2e15/users/e126dd20c41946e3ad8ffa0a90fa59e8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:52.741341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b8364a75cb44ac90ede2f5891c2e15, user_id=e126dd20c41946e3ad8ffa0a90fa59e8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:52.745238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:52.813876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:52.814180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:52.814456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:52.825567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 841/1689] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:52 2022] PUT /identity/v3/projects/84b8364a75cb44ac90ede2f5891c2e15/users/e126dd20c41946e3ad8ffa0a90fa59e8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:52.872017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] MFA Rules not processed for user `e126dd20c41946e3ad8ffa0a90fa59e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:52.873824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:52.958283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:53.018583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 849/1690] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:53.033102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:53.167645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4ab2b-f7c2-4f3e-a686-ded65d6613c3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:53.179365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:53.180955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:53.181383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:53.182224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:53.207448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:53.210789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:53.244291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 842/1691] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:44:53.340666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:53.342943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:53.378009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeae522-bd0e-4e29-8225-e36e41ec4611 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:53.389162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:53.391112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:53.392393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:53.392799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:53.501875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:53.505225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:53.573560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:53.575055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:53.597624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 850/1692] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:53.668357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db364f6-8072-4031-a509-a0e048400575 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:53.684707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:53.685721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:53.691042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:53.691636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] PATH_INFO: `/v3/projects/e95a6395c5da40eea6867858df27003b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:53.697418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95a6395c5da40eea6867858df27003b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:53.699900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:53.822935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95a6395c5da40eea6867858df27003b', 'request_id': 'req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:53.852727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95a6395c5da40eea6867858df27003b', 'request_id': 'req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:53.862981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95a6395c5da40eea6867858df27003b', 'request_id': 'req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:53.881495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:53.896806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] The token cache is being invalidate because project e95a6395c5da40eea6867858df27003b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:44:53.897330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:53.898255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:53.910295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 843/1693] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:44:53 2022] DELETE /identity/v3/projects/e95a6395c5da40eea6867858df27003b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:44:53.910425 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:53.928669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16b416-2c6a-4f0c-b228-08935ac3b51d service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:54.121338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:54.121998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:54.122280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:54.122552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] PATH_INFO: `/v3/users/191443d4f08b4ed5bc46cd5f8d1f7b49` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:54.124040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] RBAC: Authorizing `identity:delete_user(user_id=191443d4f08b4ed5bc46cd5f8d1f7b49)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:54.126573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:54.241420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191443d4f08b4ed5bc46cd5f8d1f7b49'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:54.280800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191443d4f08b4ed5bc46cd5f8d1f7b49'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:54.298577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191443d4f08b4ed5bc46cd5f8d1f7b49'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:54.299485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191443d4f08b4ed5bc46cd5f8d1f7b49'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:54.311535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-884b1551-ab64-445c-a9d2-ce3a1682886c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191443d4f08b4ed5bc46cd5f8d1f7b49'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:54.328656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 851/1694] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:44:53 2022] DELETE /identity/v3/users/191443d4f08b4ed5bc46cd5f8d1f7b49 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:56.350600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:56.352597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:56.378031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b85b06-f5bf-4f95-8b4d-65a9de1a2c86 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:56.539334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:56.541055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:56.541565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:56.542034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] PATH_INFO: `/v3/projects/39f4c19546404d0db037946415f8810e` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:56.543547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f4c19546404d0db037946415f8810e)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:56.552499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:56.685365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f4c19546404d0db037946415f8810e', 'request_id': 'req-04b84d2f-5ff2-4488-a026-754324b8457f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:56.720828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f4c19546404d0db037946415f8810e', 'request_id': 'req-04b84d2f-5ff2-4488-a026-754324b8457f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:56.742217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f4c19546404d0db037946415f8810e', 'request_id': 'req-04b84d2f-5ff2-4488-a026-754324b8457f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:56.773155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] The token cache is being invalidate because project 39f4c19546404d0db037946415f8810e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:44:56.773621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:56.775098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-04b84d2f-5ff2-4488-a026-754324b8457f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:44:56.786064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 844/1695] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:44:56 2022] DELETE /identity/v3/projects/39f4c19546404d0db037946415f8810e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:56.788127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e8b9-e979-4708-8054-e23ef9247a89 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:56.788595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e8b9-e979-4708-8054-e23ef9247a89 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:56.788861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e8b9-e979-4708-8054-e23ef9247a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:56.822467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:56.823776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2cf8d-a5fb-47b8-ac71-25549d7b57e5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:56.824132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2cf8d-a5fb-47b8-ac71-25549d7b57e5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:56.824518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2cf8d-a5fb-47b8-ac71-25549d7b57e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:56.860345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fc05e8b9-e979-4708-8054-e23ef9247a89 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:56.864523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fc05e8b9-e979-4708-8054-e23ef9247a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:56.879868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-26c2cf8d-a5fb-47b8-ac71-25549d7b57e5 None None] MFA Rules not processed for user `f417a94fed274bfbb88d23677a4877a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:56.881184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-26c2cf8d-a5fb-47b8-ac71-25549d7b57e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:57.040209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 852/1696] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:57.061247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.061406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.072841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.091800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 845/1697] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:57.113294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.114088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.114380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.118538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:57.120085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:57.139170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 853/1698] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:57.144550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.164665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb14c8e-6ebd-4153-991f-1226bc3cf590 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.178962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.179365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.179684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.179958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.185479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:44:57.186175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.198227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:57.210090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] MFA Rules not processed for user `f417a94fed274bfbb88d23677a4877a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:57.213474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:57.222317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 854/1699] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:44:57.235571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.244628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 846/1700] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:57.257421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.259733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:57.264394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a120-f4a2-4b37-a1b6-3e39fb91efa4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.280955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.285164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.285807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.286305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.289395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.299227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:57.300945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:57.385377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 855/1701] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:57 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:57.393766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.439141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c99f5-23c7-4b96-b463-717047cffd76 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.558927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.561262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.561778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.562201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.576080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67297-6923-48fe-8661-7587896ad385 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.593842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.595210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.595715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.595992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.596845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.599480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:57.657200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.659957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:57.668854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 856/1702] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:57 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:57.679394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.691387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 847/1703] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:57.716632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fca70-c1d5-4abf-b41f-59f37d1d9634 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.734497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.735397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.739142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.739649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.740368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.743481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:57.774562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 857/1704] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:57.777329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.802566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168fd7c-639c-4297-8acb-ebe562671f5f service nova] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.824247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac03cc-9018-4381-8610-bd984f73a613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.824886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac03cc-9018-4381-8610-bd984f73a613 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.825170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac03cc-9018-4381-8610-bd984f73a613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.825448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac03cc-9018-4381-8610-bd984f73a613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.831129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac03cc-9018-4381-8610-bd984f73a613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.838498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac03cc-9018-4381-8610-bd984f73a613 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:57.876933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:57.881787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 848/1705] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:57.897443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ec469-b448-4211-851c-a9e5d33f3fd8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:57.913029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a8349-aebb-40ff-a974-4eabe6c7a084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:57.918370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a8349-aebb-40ff-a974-4eabe6c7a084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:57.918894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a8349-aebb-40ff-a974-4eabe6c7a084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:57.919973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a8349-aebb-40ff-a974-4eabe6c7a084 admin admin] PATH_INFO: `/v3/projects/8e2486c25a1b4fa79700ce49a638d219/users/2e535173fdb04b56af2adb70ea5c7e25/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:57.970791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a8349-aebb-40ff-a974-4eabe6c7a084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2486c25a1b4fa79700ce49a638d219, user_id=2e535173fdb04b56af2adb70ea5c7e25, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:57.977628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a8349-aebb-40ff-a974-4eabe6c7a084 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:58.011924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 858/1706] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:57 2022] PUT /identity/v3/projects/8e2486c25a1b4fa79700ce49a638d219/users/2e535173fdb04b56af2adb70ea5c7e25/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:44:58.024222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:58.024619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:58.024779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:58.107910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] MFA Rules not processed for user `2e535173fdb04b56af2adb70ea5c7e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:58.113313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:58.183661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:58.248283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:58.248570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:58.248997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:58.308166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 849/1707] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:58.327244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:58.352806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] MFA Rules not processed for user `be2ad8e7eefe447ca1374d49696b1318`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:58.354203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:58.372214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:58.552765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 859/1708] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:44:58.567554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:58.571506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:58.573905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815b13a-72fd-4043-823c-9115ae5827d1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:58.598190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:58.600244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:58.600635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:58.600824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:58.601903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:58.606528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:58.688220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 850/1709] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:44:58 2022] POST /identity/v3/projects => generated 381 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:58.689844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:58.823807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ae51f-86ab-489a-becf-9062c06abc8c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:58.844289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c95b33-b967-4f6d-a0fc-c01f11fe9b8e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:58.855099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:58.855929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:58.856279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:58.856968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:58.857356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:58.862690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:58.927911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 851/1710] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:44:58 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:44:58.929827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:58.938757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:58.988443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:58.989109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:58.989566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:58.992919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.093229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:59.102771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:44:59.131907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455cb38-f71d-438c-b8f3-170ccabce65a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:59.136701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:59.138054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:59.151003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:59.151342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.153829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:44:59.155415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:59.157698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:59.184189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 860/1711] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:59.186069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:59.210511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8cff-fcf8-4435-949e-4b7ad27fd480 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:59.226165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:59.226394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:59.226646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:59.226863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.227548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:59.229830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:59.250123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 861/1712] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:59.255510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:59.261305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:59.266936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:59.282652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eee67-e8fc-488a-9077-52bc98496f10 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:59.295391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 852/1713] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:44:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:44:59.305510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:59.306276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:59.306598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:59.306888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] PATH_INFO: `/v3/projects/f181fcf85188426181c03fdcad1a8424/users/571388cad645410285982698f0f69690/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.364470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f181fcf85188426181c03fdcad1a8424, user_id=571388cad645410285982698f0f69690, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:59.371814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:59.400905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 862/1714] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:59 2022] PUT /identity/v3/projects/f181fcf85188426181c03fdcad1a8424/users/571388cad645410285982698f0f69690/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:44:59.418905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:59.454706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6b9e4-32ad-4e4d-8ba7-b5120126d89b service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:59.680649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d740d8-f8eb-404a-aafb-55c04407e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:59.683231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d740d8-f8eb-404a-aafb-55c04407e302 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:59.683769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d740d8-f8eb-404a-aafb-55c04407e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:59.684262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d740d8-f8eb-404a-aafb-55c04407e302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.689604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d740d8-f8eb-404a-aafb-55c04407e302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:59.693421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d740d8-f8eb-404a-aafb-55c04407e302 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:59.756288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 853/1715] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:44:59.761573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:44:59.772855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2df86e-95fa-47c7-bd89-dd0a94bf1334 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:44:59.792701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818acc89-16ce-470f-bffd-687e20efbc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:44:59.793684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818acc89-16ce-470f-bffd-687e20efbc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:59.794223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818acc89-16ce-470f-bffd-687e20efbc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:59.794703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818acc89-16ce-470f-bffd-687e20efbc0a admin admin] PATH_INFO: `/v3/projects/f181fcf85188426181c03fdcad1a8424/users/571388cad645410285982698f0f69690/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.800422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818acc89-16ce-470f-bffd-687e20efbc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f181fcf85188426181c03fdcad1a8424, user_id=571388cad645410285982698f0f69690, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:44:59.805386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818acc89-16ce-470f-bffd-687e20efbc0a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:44:59.840402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 863/1716] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:44:59 2022] PUT /identity/v3/projects/f181fcf85188426181c03fdcad1a8424/users/571388cad645410285982698f0f69690/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:44:59.840640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:44:59.841073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:44:59.841422 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:44:59.920852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] MFA Rules not processed for user `571388cad645410285982698f0f69690`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:44:59.922640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:00.011991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f8c47-4caa-4b52-910c-54c53e999124 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:00.012536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f8c47-4caa-4b52-910c-54c53e999124 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:00.012731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f8c47-4caa-4b52-910c-54c53e999124 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:00.015199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 864/1717] 10.222.0.55 () {54 vars in 912 bytes} [Sat Sep 24 19:45:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:45:00.023095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d813afe-4222-4775-81e5-da33533c108b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:00.023533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d813afe-4222-4775-81e5-da33533c108b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:00.024207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d813afe-4222-4775-81e5-da33533c108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:00.134946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 854/1718] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:44:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:00.237432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2d813afe-4222-4775-81e5-da33533c108b None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:00.237432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2d813afe-4222-4775-81e5-da33533c108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:00.237996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:00.273716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:00.456205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cc7a1-ba33-482c-906d-5c387b01c26a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:00.467531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:00.469903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 865/1719] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:45:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:00.476150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:00.477708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:00.480403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:00.483451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:00.488163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:00.488609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:00.526171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:00.527843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:00.553751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 855/1720] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:00.575983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:00.579044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:00.594598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] MFA Rules not processed for user `444719e34bf2457b9a58e182ef7fbd79`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:00.596098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:00.603854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb4ac-2154-4d42-a24d-8ac4840e5cd5 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:00.615941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:00.616651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:00.617053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:00.617476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:00.646833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:00.648947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:00.673304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 856/1721] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:00.886138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 866/1722] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:00.895874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:00.926439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:00.928270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:00.972084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5b47c-6825-4829-bd9f-4a55aec33cb6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:00.994443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bca38d-28cc-4fab-99c6-0154caa9ab7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:00.997054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bca38d-28cc-4fab-99c6-0154caa9ab7c service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:00.997229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bca38d-28cc-4fab-99c6-0154caa9ab7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:01.001352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bca38d-28cc-4fab-99c6-0154caa9ab7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:01.095871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:01.198783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9b962-c08a-47ac-b61b-c5cad0b70df4 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:01.224491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263b4c9-493e-4c83-8dc5-df57441db0ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:01.227377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263b4c9-493e-4c83-8dc5-df57441db0ec service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:01.227377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263b4c9-493e-4c83-8dc5-df57441db0ec service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:01.229474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263b4c9-493e-4c83-8dc5-df57441db0ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:01.252456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bca38d-28cc-4fab-99c6-0154caa9ab7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:01.256724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bca38d-28cc-4fab-99c6-0154caa9ab7c service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:01.264140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263b4c9-493e-4c83-8dc5-df57441db0ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:01.267013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263b4c9-493e-4c83-8dc5-df57441db0ec service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:01.298488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 867/1723] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:01.315608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 857/1724] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:02.988175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:02.990189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:02.990656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:03.028545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] MFA Rules not processed for user `be2ad8e7eefe447ca1374d49696b1318`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:03.030083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:03.135551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 868/1725] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:03.467626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabc94d-74ce-4750-8669-407e82420bc0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:03.468193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabc94d-74ce-4750-8669-407e82420bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:03.468193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabc94d-74ce-4750-8669-407e82420bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:03.543518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1eabc94d-74ce-4750-8669-407e82420bc0 None None] MFA Rules not processed for user `89b20ec0ede5470eaa33e1ea934991d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:03.547936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1eabc94d-74ce-4750-8669-407e82420bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:03.791081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:03.793087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:03.817810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 858/1726] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:03.825349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e364-8c2e-4336-ac87-9a2be699ff83 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:03.826001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e364-8c2e-4336-ac87-9a2be699ff83 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:03.826589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e364-8c2e-4336-ac87-9a2be699ff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:03.899246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-aee6e364-8c2e-4336-ac87-9a2be699ff83 None None] MFA Rules not processed for user `89b20ec0ede5470eaa33e1ea934991d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:03.906531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-aee6e364-8c2e-4336-ac87-9a2be699ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:03.930718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 859/1727] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:04.037518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:04.037806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:04.038080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:04.087316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] MFA Rules not processed for user `f417a94fed274bfbb88d23677a4877a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:04.088668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:04.107769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:04.120365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bfd63-c0c8-4f3d-8d2b-70680631cea0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:04.132144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:04.133478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:04.134065 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:04.134635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:04.138904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:04.165923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:04.169523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54f049-e26e-4043-abc9-344bd97f5957 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:04.209973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 860/1728] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:04.211144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:04.229992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 869/1729] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:04.235365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cbc3e-fed9-45a9-be7f-7bfec1814485 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:04.247040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab846-c0a8-4469-9b11-ce1dd62d9af0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:04.247775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab846-c0a8-4469-9b11-ce1dd62d9af0 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:04.248062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab846-c0a8-4469-9b11-ce1dd62d9af0 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:04.248333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab846-c0a8-4469-9b11-ce1dd62d9af0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:04.272152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cab846-c0a8-4469-9b11-ce1dd62d9af0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:04.274361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cab846-c0a8-4469-9b11-ce1dd62d9af0 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:04.294787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 861/1730] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:04.378307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74324a0-b85a-4563-a83e-290258e10d06 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:04.379163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74324a0-b85a-4563-a83e-290258e10d06 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:04.379980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74324a0-b85a-4563-a83e-290258e10d06 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:04.382669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 870/1731] 10.222.0.55 () {54 vars in 912 bytes} [Sat Sep 24 19:45:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:45:04.390082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:04.390356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:04.391454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:04.454844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:04.462195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:04.489698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 862/1732] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:45:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:04.546355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74324a0-b85a-4563-a83e-290258e10d06 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:04.550497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e74324a0-b85a-4563-a83e-290258e10d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:04.574937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74324a0-b85a-4563-a83e-290258e10d06 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:04.586653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16751-2f6d-41bc-b14f-be66c8f044e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:04.587558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16751-2f6d-41bc-b14f-be66c8f044e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:04.589116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16751-2f6d-41bc-b14f-be66c8f044e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:04.589557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16751-2f6d-41bc-b14f-be66c8f044e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:04.625660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16751-2f6d-41bc-b14f-be66c8f044e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:04.627853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16751-2f6d-41bc-b14f-be66c8f044e5 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:04.653551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 871/1733] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:05.244294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:05.246244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:05.492611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a6674-e4c3-4a01-833a-87a2ec1bb04b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:05.493095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a6674-e4c3-4a01-833a-87a2ec1bb04b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:05.493197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a6674-e4c3-4a01-833a-87a2ec1bb04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:05.591858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff89bf-0b78-457b-88f1-aaa0e8760b3b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:05.605196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a15a6674-e4c3-4a01-833a-87a2ec1bb04b None None] MFA Rules not processed for user `6974a4c8fffd4bc2933e754770cb912e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:05.607274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a15a6674-e4c3-4a01-833a-87a2ec1bb04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:05.610230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:05.611493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:05.611951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:05.612360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] PATH_INFO: `/v3/users/4cd5ac8eedef455c9470e4c6580d70ef` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:05.619053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd5ac8eedef455c9470e4c6580d70ef)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:05.622794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:05.746096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:05.807409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd5ac8eedef455c9470e4c6580d70ef'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:05.835441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd5ac8eedef455c9470e4c6580d70ef'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:05.837658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd5ac8eedef455c9470e4c6580d70ef'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:05.839653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd5ac8eedef455c9470e4c6580d70ef'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:05.853180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 872/1734] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:05.862507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd5ac8eedef455c9470e4c6580d70ef'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:05.917427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 863/1735] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:05 2022] DELETE /identity/v3/users/4cd5ac8eedef455c9470e4c6580d70ef => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:05.923792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:05.924325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:05.924784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:05.994706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:05.999735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:06.214161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:06.217907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:06.257482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 873/1736] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:06.269798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:06.271213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:06.284989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:06.432487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ae894-a882-40ab-8cc1-d79a634b8cdc admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:06.447118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:06.447795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:06.448078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:06.448352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] PATH_INFO: `/v3/users/2e535173fdb04b56af2adb70ea5c7e25` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:06.456483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e535173fdb04b56af2adb70ea5c7e25)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:06.458474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:06.536885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d961-62a5-4060-a758-3f6d5a82241d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:06.564178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:06.564820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:06.565103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:06.565377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:06.625723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:06.627608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:06.661908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 874/1737] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:06.662240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:06.664101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:06.684145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e535173fdb04b56af2adb70ea5c7e25'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:06.702111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e535173fdb04b56af2adb70ea5c7e25'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:06.703662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e535173fdb04b56af2adb70ea5c7e25'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:06.705579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e535173fdb04b56af2adb70ea5c7e25'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:06.731582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9093d2d9-2165-465a-98a9-17ebaac053e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e535173fdb04b56af2adb70ea5c7e25'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:06.797652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 864/1738] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:06 2022] DELETE /identity/v3/users/2e535173fdb04b56af2adb70ea5c7e25 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:06.802804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:06.803223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:06.803910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:06.848901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:06.850468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:06.864527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4a05-f6a4-4f5b-9318-62c891dbfa86 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:06.897173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:07.009130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 865/1739] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:07.032946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:07.033678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:07.033960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:07.034232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:07.038985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:07.046722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:07.048506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:07.149026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:07.190755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7ee4e-b432-4372-959f-ce122dedd327 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:07.197343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:07.200265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:07.201278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e304b1-a050-49a4-b7d1-d973971a3ea4 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:07.201797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:07.202193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:07.202467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:07.229060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:07.229534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 875/1740] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:07.232185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:07.251214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 866/1741] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:07.991289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:07.991818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:07.991905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:08.082270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] MFA Rules not processed for user `444719e34bf2457b9a58e182ef7fbd79`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:08.084081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:08.120217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:08.121971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:08.176976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844624-f9b1-4219-8db1-f616ea308bbb service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:08.196276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:08.197063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:08.197936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:08.197936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] PATH_INFO: `/v3/projects/be93e774048a472abdc3ce1bf16e74a1` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:08.204460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be93e774048a472abdc3ce1bf16e74a1)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:08.207186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:08.294748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 876/1742] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:08.400985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be93e774048a472abdc3ce1bf16e74a1', 'request_id': 'req-739a9a5c-e1ad-4887-b143-e18a2f789088'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.426824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be93e774048a472abdc3ce1bf16e74a1', 'request_id': 'req-739a9a5c-e1ad-4887-b143-e18a2f789088'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.435345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be93e774048a472abdc3ce1bf16e74a1', 'request_id': 'req-739a9a5c-e1ad-4887-b143-e18a2f789088'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.462259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] The token cache is being invalidate because project be93e774048a472abdc3ce1bf16e74a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:08.462763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.464554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.472507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:08.473104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 867/1743] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:08 2022] DELETE /identity/v3/projects/be93e774048a472abdc3ce1bf16e74a1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:08.486897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec075fd3-bb39-4b61-ad23-386de08ce63e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:08.583801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:08.583801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:08.669448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:08.708810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:08.710550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:08.714206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:08.714783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] PATH_INFO: `/v3/users/e126dd20c41946e3ad8ffa0a90fa59e8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:08.717633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e126dd20c41946e3ad8ffa0a90fa59e8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:08.725543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:08.844599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a9a5c-e1ad-4887-b143-e18a2f789088 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:08.854062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:08.855140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:08.855651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:08.856189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] PATH_INFO: `/v3/projects/8e2486c25a1b4fa79700ce49a638d219` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:08.858686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2486c25a1b4fa79700ce49a638d219)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:08.861670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:08.864504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e126dd20c41946e3ad8ffa0a90fa59e8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.885471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e126dd20c41946e3ad8ffa0a90fa59e8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.902190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e126dd20c41946e3ad8ffa0a90fa59e8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.906745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e126dd20c41946e3ad8ffa0a90fa59e8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.936567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e126dd20c41946e3ad8ffa0a90fa59e8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:08.959272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 877/1744] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:08 2022] DELETE /identity/v3/users/e126dd20c41946e3ad8ffa0a90fa59e8 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:08.983141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2486c25a1b4fa79700ce49a638d219', 'request_id': 'req-3c1910c5-005e-4519-a076-a79f5b2ca4a9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.003505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2486c25a1b4fa79700ce49a638d219', 'request_id': 'req-3c1910c5-005e-4519-a076-a79f5b2ca4a9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.014494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:09.018345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2486c25a1b4fa79700ce49a638d219', 'request_id': 'req-3c1910c5-005e-4519-a076-a79f5b2ca4a9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.080448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] The token cache is being invalidate because project 8e2486c25a1b4fa79700ce49a638d219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:09.081429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.081784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.083594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 868/1745] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:08 2022] DELETE /identity/v3/projects/8e2486c25a1b4fa79700ce49a638d219 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:09.088049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:09.092213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:09.283668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1993804-6ed1-4e8a-b569-b0b63caa41b5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:09.293780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:09.293780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:09.293780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:09.293780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] PATH_INFO: `/v3/users/571388cad645410285982698f0f69690` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:09.295490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571388cad645410285982698f0f69690)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:09.298118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:09.356438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571388cad645410285982698f0f69690'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.373552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571388cad645410285982698f0f69690'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.389029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571388cad645410285982698f0f69690'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.389759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571388cad645410285982698f0f69690'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.393221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:09.410287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-952723e9-4c4e-4c7d-a242-1ae40381a053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571388cad645410285982698f0f69690'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:09.427048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 878/1746] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:09 2022] DELETE /identity/v3/users/571388cad645410285982698f0f69690 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:10.069833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:10.081225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:10.243151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1910c5-005e-4519-a076-a79f5b2ca4a9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:10.389692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:10.390278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:10.396785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:10.396785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] PATH_INFO: `/v3/projects/84b8364a75cb44ac90ede2f5891c2e15` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:10.396785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b8364a75cb44ac90ede2f5891c2e15)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:10.406089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:10.516700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b8364a75cb44ac90ede2f5891c2e15', 'request_id': 'req-c9b40093-50fe-4fbc-ba15-1b528cb35a60'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:10.532824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b8364a75cb44ac90ede2f5891c2e15', 'request_id': 'req-c9b40093-50fe-4fbc-ba15-1b528cb35a60'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:10.552464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b8364a75cb44ac90ede2f5891c2e15', 'request_id': 'req-c9b40093-50fe-4fbc-ba15-1b528cb35a60'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:10.580002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] The token cache is being invalidate because project 84b8364a75cb44ac90ede2f5891c2e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:10.580366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:10.581315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9b40093-50fe-4fbc-ba15-1b528cb35a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:10.583528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 869/1747] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:10 2022] DELETE /identity/v3/projects/84b8364a75cb44ac90ede2f5891c2e15 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:10.588615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:10.589797 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:10.590576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:10.672784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:10.677823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:10.801424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 879/1748] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:10.806929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:10.807310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:10.807788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:10.856129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:10.857788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:10.919297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 870/1749] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:10.922221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:10.942722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb702a3-d26b-4426-8a45-ea3920f2789f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:10.967023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:10.968248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:10.969074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:10.970837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:10.971594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:10.973990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:10.981421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.011802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.014125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:11.032599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a092cd-b6de-4094-9436-3095f6f07ea3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.039806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.040941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.040941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.041216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] PATH_INFO: `/v3/projects/f181fcf85188426181c03fdcad1a8424` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.042521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f181fcf85188426181c03fdcad1a8424)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.045229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.068828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 880/1750] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:11.069906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.096678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:11.098459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82f015-9870-4c2f-b3a8-78b2731fbe73 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.112024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.112654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.112927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.113189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.113829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.116249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.217607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 881/1751] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:11.221646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.242627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f181fcf85188426181c03fdcad1a8424', 'request_id': 'req-7767be7a-96f7-44d5-be32-1e37441f0b79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:11.267286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f181fcf85188426181c03fdcad1a8424', 'request_id': 'req-7767be7a-96f7-44d5-be32-1e37441f0b79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:11.298868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f181fcf85188426181c03fdcad1a8424', 'request_id': 'req-7767be7a-96f7-44d5-be32-1e37441f0b79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:11.339604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] The token cache is being invalidate because project f181fcf85188426181c03fdcad1a8424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:11.339937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:11.340655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7767be7a-96f7-44d5-be32-1e37441f0b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:11.345282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 871/1752] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:11 2022] DELETE /identity/v3/projects/f181fcf85188426181c03fdcad1a8424 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:11.350364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31e93a-f1d8-447b-9616-0481fa86d6c2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.350642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31e93a-f1d8-447b-9616-0481fa86d6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.351041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31e93a-f1d8-447b-9616-0481fa86d6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.379156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb62fa7-5b0d-4748-ac2c-4b84a133ea8b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.385960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.386855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.388021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.388512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.389365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.391664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.410166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-fb31e93a-f1d8-447b-9616-0481fa86d6c2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:11.411470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fb31e93a-f1d8-447b-9616-0481fa86d6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:11.422279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 882/1753] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:11.424329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.427095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:11.454332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 872/1754] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:11.454703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c32d3-93e2-4e85-bab4-3e223399018a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.456866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.457162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.457490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.461006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.461892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.462304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.462640 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.465001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.467028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.494159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 883/1755] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:11.499507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.526051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39a9c9-6bbb-4584-82d8-4581af14ddc1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.535240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.535990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.536361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.536697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.537424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.540294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:11.542004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.542515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:11.575445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 873/1757] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:11.576377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 884/1756] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:11.577295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.577963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.616492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7dfc69-62d5-4379-8ca5-5bda5f2b6837 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.622459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba70124-e078-469c-af90-25853396de7c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.633241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.633241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.633241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.633623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.634425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:11.634425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.635104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067177a-7a40-43f7-a5c7-bf122a9fb378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.639168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067177a-7a40-43f7-a5c7-bf122a9fb378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.639561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067177a-7a40-43f7-a5c7-bf122a9fb378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.639775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067177a-7a40-43f7-a5c7-bf122a9fb378 admin admin] PATH_INFO: `/v3/projects/d75190e5db3e4078a5077ccd02e38329/users/999fab069ac54929a957cfe640f5cbbe/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.645075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.682120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067177a-7a40-43f7-a5c7-bf122a9fb378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75190e5db3e4078a5077ccd02e38329, user_id=999fab069ac54929a957cfe640f5cbbe, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.688552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 874/1758] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:11.690478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067177a-7a40-43f7-a5c7-bf122a9fb378 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.692683 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.708825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:11.709515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810aca89-3d14-4e61-8281-834767b2b2d6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:11.730359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 885/1759] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:11 2022] PUT /identity/v3/projects/d75190e5db3e4078a5077ccd02e38329/users/999fab069ac54929a957cfe640f5cbbe/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:11.733381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.733664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.733932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.797263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] MFA Rules not processed for user `999fab069ac54929a957cfe640f5cbbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:11.798881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:11.807103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:11.809993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:11.810935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:11.811248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:11.813175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:11.818554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:11.929888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 875/1760] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/projects => generated 369 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:45:11.931325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 886/1761] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:11.947473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.948013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:11.984933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c00e3-0e09-4e4a-af9f-6dc6bf715085 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.103876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d929db-3181-4d3f-b1f9-855c668927f7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.124218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb17b1-7568-436e-91ee-a6a6642c5db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.125005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb17b1-7568-436e-91ee-a6a6642c5db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.125278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb17b1-7568-436e-91ee-a6a6642c5db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.125547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb17b1-7568-436e-91ee-a6a6642c5db1 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.126201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb17b1-7568-436e-91ee-a6a6642c5db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.128199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb17b1-7568-436e-91ee-a6a6642c5db1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:12.160257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:12.186357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 876/1762] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:12.187937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.191350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.192120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.227179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.227881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.228162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.228529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.230089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.239301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:12.354882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:12.360520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:12.389798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 887/1763] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:11 2022] POST /identity/v3/projects => generated 379 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:12.393167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:12.616537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eaca6-d1e6-4bd1-a593-d144fefbb042 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.638693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 877/1764] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:45:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:12.640008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:12.641687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.645312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.645619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.645900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.647647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.657302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:12.739307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 888/1765] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:12.740842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a07f3a-21e6-4286-8bcc-aa31ae24d84a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.748882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:12.792497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e473f1-48fd-43a1-90a5-fe68f1ebb781 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.793583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.794255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.801415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.801788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.802508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.804712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:12.808922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.809554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.810000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.810283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.811541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.816334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:12.857545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:12.875558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 878/1766] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:12 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:12.880412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:12.887001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 889/1767] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:12.888267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:12.893253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827579-61ce-40d7-8028-c7fa9751fe03 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.900161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.901116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.901572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.902038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.902881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.916533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:12.928346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69629474-744a-42f5-9c21-daa873217d5b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:12.934367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:12.935037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:12.935358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:12.935663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] PATH_INFO: `/v3/projects/2455ea6358b2431784ce9de2008390d5/users/ca64cdd1ba6e47f1b1aed3a7fadd4c56/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:12.948816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 879/1768] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:12.962630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:12.978540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2455ea6358b2431784ce9de2008390d5, user_id=ca64cdd1ba6e47f1b1aed3a7fadd4c56, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:12.982378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:13.003468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 890/1769] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:12 2022] PUT /identity/v3/projects/2455ea6358b2431784ce9de2008390d5/users/ca64cdd1ba6e47f1b1aed3a7fadd4c56/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:13.011214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f9e85-c0bc-4a2f-ac6b-cdffe64b2043 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:13.015895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:13.018132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:13.152249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c343a-fa4b-47b0-9ec9-cd6df9979d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:13.152943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c343a-fa4b-47b0-9ec9-cd6df9979d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.153251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c343a-fa4b-47b0-9ec9-cd6df9979d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.153594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c343a-fa4b-47b0-9ec9-cd6df9979d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.154279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c343a-fa4b-47b0-9ec9-cd6df9979d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:13.158669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b2633-d855-42f7-8581-af8d243bfdf5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:13.161000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c343a-fa4b-47b0-9ec9-cd6df9979d4b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:13.182200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:13.182200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.182200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.182200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.183287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 880/1770] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:13.185130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.185573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.185989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.237564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] MFA Rules not processed for user `ca64cdd1ba6e47f1b1aed3a7fadd4c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:13.239064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:13.276355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:13.277149 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:13.290595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:13.322885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 891/1771] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:45:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:45:13.325387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:13.383733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 881/1772] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:13.394386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:13.396397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:13.412185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127692c-1c0a-4870-be6f-c936dd7ea0cb service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:13.449026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f1abb-2b61-4ef9-bebf-d650d0870834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:13.451370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f1abb-2b61-4ef9-bebf-d650d0870834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.456407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f1abb-2b61-4ef9-bebf-d650d0870834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.458853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f1abb-2b61-4ef9-bebf-d650d0870834 admin admin] PATH_INFO: `/v3/projects/c2749459fa9a4059a6bd3edf09ccad39/users/3eb8f4e306314c8ba2ccd4812fdef519/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.511156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f1abb-2b61-4ef9-bebf-d650d0870834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2749459fa9a4059a6bd3edf09ccad39, user_id=3eb8f4e306314c8ba2ccd4812fdef519, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:13.522311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f1abb-2b61-4ef9-bebf-d650d0870834 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:13.579752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 892/1773] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:13 2022] PUT /identity/v3/projects/c2749459fa9a4059a6bd3edf09ccad39/users/3eb8f4e306314c8ba2ccd4812fdef519/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:13.607169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053bba5-49e9-4b18-a374-12bf4f7ca617 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.607467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053bba5-49e9-4b18-a374-12bf4f7ca617 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.607796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053bba5-49e9-4b18-a374-12bf4f7ca617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.654383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0316cf5-1466-43ac-83c4-40644967fd4c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:13.681465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e053bba5-49e9-4b18-a374-12bf4f7ca617 None None] MFA Rules not processed for user `3eb8f4e306314c8ba2ccd4812fdef519`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:13.683164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e053bba5-49e9-4b18-a374-12bf4f7ca617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:13.766392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:13.809414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:13.810053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.810328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.810596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.883378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 893/1774] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:13.893596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:13.893927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:13.894161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:13.961607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] MFA Rules not processed for user `6974a4c8fffd4bc2933e754770cb912e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:13.963758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:13.992366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:13.996931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:14.028316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 882/1775] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:14.029020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:14.081260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 894/1776] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:14.082839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:14.173050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2d5b-fff7-4971-aecd-fe1251f3a83b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:14.181453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:14.196930 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:14.198033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:14.199266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] PATH_INFO: `/v3/users/ca64cdd1ba6e47f1b1aed3a7fadd4c56` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:14.200865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:14.207046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca64cdd1ba6e47f1b1aed3a7fadd4c56)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:14.212404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:14.309196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a58ffb-c1ed-4fc4-85f8-8c66568c145d service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:14.330951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:14.331890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:14.332283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:14.332706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:14.333660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:14.336785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:14.389707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca64cdd1ba6e47f1b1aed3a7fadd4c56'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:14.412742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca64cdd1ba6e47f1b1aed3a7fadd4c56'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:14.421574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:14.429957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca64cdd1ba6e47f1b1aed3a7fadd4c56'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:14.431587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca64cdd1ba6e47f1b1aed3a7fadd4c56'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:14.440382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 883/1777] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:14 2022] POST /identity/v3/projects => generated 377 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:14.446470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:14.452397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-72e255ab-577b-4bfa-915e-ef8b565b6b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca64cdd1ba6e47f1b1aed3a7fadd4c56'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:14.478999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 895/1778] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:14 2022] DELETE /identity/v3/users/ca64cdd1ba6e47f1b1aed3a7fadd4c56 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:14.485322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:14.485957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:14.486611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:14.602403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:14.607292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:14.689129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489460b-aa85-4ba4-9438-2121bec24122 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:14.717393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 896/1779] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:14.792862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:14.794934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:14.822988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:14.824165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:14.829212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:14.829761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:14.830613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:14.833141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:14.886431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 884/1780] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:14 2022] POST /identity/v3/users => generated 295 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:14.887738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:14.891806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:14.915083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc26db3-c3fc-4dde-9b9f-aa9feaf4c7b8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:14.919647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b10295-eebe-42d5-9f04-5f5b34504883 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:14.926254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:14.926981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:14.927267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:14.931794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:14.940342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:14.941154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:14.941576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:14.941927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:14.942796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:14.944706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:14.952459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:14.956860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:14.979018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 885/1781] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:14.980005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:14.993757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 897/1782] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:15.002487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea9087-bc9d-4fee-b69f-e7c3976b458f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:15.022468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:15.026325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:15.026325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:15.026325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] PATH_INFO: `/v3/projects/cafb82089d6c4ee5aefd8d8e0dc37cef/users/58e56f2c29a44fb988b83076b5a69e08/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:15.062015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafb82089d6c4ee5aefd8d8e0dc37cef, user_id=58e56f2c29a44fb988b83076b5a69e08, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:15.066322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:15.092675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 886/1783] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:14 2022] PUT /identity/v3/projects/cafb82089d6c4ee5aefd8d8e0dc37cef/users/58e56f2c29a44fb988b83076b5a69e08/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:15.097402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:15.113625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4150-bb76-45b4-958b-6d1b4dc561b6 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:15.309391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aab5a-bd6a-461f-9762-12dab322bef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:15.311098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aab5a-bd6a-461f-9762-12dab322bef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:15.313021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aab5a-bd6a-461f-9762-12dab322bef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:15.313484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aab5a-bd6a-461f-9762-12dab322bef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:15.316358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aab5a-bd6a-461f-9762-12dab322bef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:15.320785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aab5a-bd6a-461f-9762-12dab322bef1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:15.361335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 898/1784] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:15.361560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:15.396563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14a4c-4000-4c4c-9473-0cc6c994c956 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:15.415379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:15.416927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:15.417544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:15.419254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] PATH_INFO: `/v3/projects/cafb82089d6c4ee5aefd8d8e0dc37cef/users/58e56f2c29a44fb988b83076b5a69e08/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:15.423813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafb82089d6c4ee5aefd8d8e0dc37cef, user_id=58e56f2c29a44fb988b83076b5a69e08, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:15.426945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:15.497952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:15.498267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:15.498517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:15.504113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 887/1785] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:15 2022] PUT /identity/v3/projects/cafb82089d6c4ee5aefd8d8e0dc37cef/users/58e56f2c29a44fb988b83076b5a69e08/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:15.595329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] MFA Rules not processed for user `58e56f2c29a44fb988b83076b5a69e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:15.596934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:15.612227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:15.762971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 899/1786] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:15.776619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:15.780150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:15.808847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:15.914732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:15.917077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:16.091928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626e8e0-bf50-4e59-a282-5979092ed7c7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:16.103364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998dae-88d1-4c60-96b5-7b6acfb4a8d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:16.104034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998dae-88d1-4c60-96b5-7b6acfb4a8d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:16.104312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998dae-88d1-4c60-96b5-7b6acfb4a8d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:16.104576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998dae-88d1-4c60-96b5-7b6acfb4a8d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:16.141120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b6d16-23b2-49c7-a65c-437a0d8840ea None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:16.157652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:16.158649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:16.158927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:16.159253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] PATH_INFO: `/v3/projects/2455ea6358b2431784ce9de2008390d5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:16.162658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2455ea6358b2431784ce9de2008390d5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:16.165407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:16.179793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25998dae-88d1-4c60-96b5-7b6acfb4a8d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:16.181619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25998dae-88d1-4c60-96b5-7b6acfb4a8d5 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:16.237064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 888/1787] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:16.242491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6dc37-02ce-420f-9e40-0caaab6a6f25 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:16.243388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6dc37-02ce-420f-9e40-0caaab6a6f25 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:16.245758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6dc37-02ce-420f-9e40-0caaab6a6f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:16.311958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-dad6dc37-02ce-420f-9e40-0caaab6a6f25 None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:16.318227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dad6dc37-02ce-420f-9e40-0caaab6a6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:16.421184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2455ea6358b2431784ce9de2008390d5', 'request_id': 'req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:16.455551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2455ea6358b2431784ce9de2008390d5', 'request_id': 'req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:16.464736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2455ea6358b2431784ce9de2008390d5', 'request_id': 'req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:16.491975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] The token cache is being invalidate because project 2455ea6358b2431784ce9de2008390d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:16.492271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:16.494584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:16.496821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 900/1788] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:15 2022] DELETE /identity/v3/projects/2455ea6358b2431784ce9de2008390d5 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:16.503781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:16.509183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:16.647106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 889/1789] 10.222.0.65 () {58 vars in 1015 bytes} [Sat Sep 24 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:16.653385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a0d90-ba69-4840-bd43-272d190ee4ee None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:16.653713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a0d90-ba69-4840-bd43-272d190ee4ee None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:16.653988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a0d90-ba69-4840-bd43-272d190ee4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:16.662404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:16.705915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07d0c-8323-48d4-bbb7-5d739e4d4a9d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:16.724231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:16.724997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:16.738206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:16.738625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:16.744976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-268a0d90-ba69-4840-bd43-272d190ee4ee None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:16.748038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-268a0d90-ba69-4840-bd43-272d190ee4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:16.751959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:17.059787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:17.062249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:17.090431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 901/1790] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:17.091603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:17.104330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:17.121629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ac536-c73a-4b9b-b6d6-28159af9d2a0 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:17.130422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:17.131150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:17.131462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:17.131735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:17.143634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:17.146597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:17.162221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 902/1791] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:17.163011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:17.195642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f479d-c8aa-49f4-a99d-abe0bb2de7a2 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:17.200121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:17.200742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:17.201020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:17.201291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:17.217316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 890/1792] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:17.219562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:17.220041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:17.222972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:17.247906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:17.249374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:17.300417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:17.305161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:17.317656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 903/1793] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:17.358453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:17.359554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 891/1794] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:17.393749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d31251-8eca-47ba-bef7-a60b2cdaec14 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:17.411307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:17.414533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:17.415925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:17.420173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:17.423211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:17.423773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:17.428492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:17.490161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 904/1795] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:45:17.494520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:17.509079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d5720-0a4c-46ee-9050-8790ead1b9f4 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:17.551633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:17.552203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:17.552473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:17.552921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:17.556191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:17.564797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:17.656427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 892/1796] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:17 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:17.661728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:17.694686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfacdb-a976-430d-9700-1fb0db7ed956 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:18.018697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:18.023337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:18.025229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:18.027909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:18.029919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:18.045184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:18.186293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 905/1797] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:17 2022] POST /identity/v3/users => generated 289 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:18.189409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:18.208431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1176d33-c080-457b-a5ee-55d79b0a13fb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:18.234902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:18.238108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:18.239757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:18.241147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:18.244482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:18.250274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:18.318567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:18.331793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 893/1798] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:18.337549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803ed87-c77f-48f5-baa2-920d6d717ab0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:18.373854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22565a3-067f-447a-89b0-b651f4edc872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:18.374512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22565a3-067f-447a-89b0-b651f4edc872 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:18.374836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22565a3-067f-447a-89b0-b651f4edc872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:18.375105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22565a3-067f-447a-89b0-b651f4edc872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:18.381705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22565a3-067f-447a-89b0-b651f4edc872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:18.386957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22565a3-067f-447a-89b0-b651f4edc872 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:18.450758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:18.466932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 906/1799] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:18.479105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be707af-dcaa-4fff-804c-98d91d73588f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:18.502535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:18.503729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:18.506198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:18.506716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] PATH_INFO: `/v3/projects/b16c7408b63043a094046a026dda098d/users/29dc3c2da9064c0181a81accff957a13/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:18.593330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16c7408b63043a094046a026dda098d, user_id=29dc3c2da9064c0181a81accff957a13, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:18.618410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:18.644094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:18.682706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 894/1800] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:18 2022] PUT /identity/v3/projects/b16c7408b63043a094046a026dda098d/users/29dc3c2da9064c0181a81accff957a13/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:18.695739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:18.696031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:18.697675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:18.783581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] MFA Rules not processed for user `29dc3c2da9064c0181a81accff957a13`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:18.785188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:18.866087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:18.991857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 907/1801] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:18.996705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:19.130378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5127cb-3738-4d52-8c32-63f017d57a92 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:19.136576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:19.137241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:19.137532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:19.138001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:19.145633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:19.147927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:19.227474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 895/1802] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:18 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:19.229202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:19.244603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383af18-bb78-4ceb-ab2c-47679ff9ec6d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:19.430007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:19.431687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:19.431687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:19.431687 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:19.432040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:19.434646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:19.476199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 908/1803] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:19 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:19.477114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:19.511935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2541ac-a7ab-4c53-9cb2-0a74f07ff369 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:19.526501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:19.527490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:19.528044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:19.529157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:19.530095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:19.532413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:19.586787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 896/1804] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:19.593493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:19.621877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c91de-9a3c-42f5-ac20-392b228364e8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:19.673968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80feded-ab1e-4b48-8fae-fa0fa570e43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:19.675012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80feded-ab1e-4b48-8fae-fa0fa570e43e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:19.675147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80feded-ab1e-4b48-8fae-fa0fa570e43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:19.675521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80feded-ab1e-4b48-8fae-fa0fa570e43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:19.676498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80feded-ab1e-4b48-8fae-fa0fa570e43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:19.678155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80feded-ab1e-4b48-8fae-fa0fa570e43e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:19.712557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:19.713189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 909/1805] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:19.731056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0871a0f-c1b3-4d61-91eb-7b5d6b2752b1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:19.761061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:19.761812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:19.762097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:19.762365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] PATH_INFO: `/v3/projects/2b37f29e0cc94671847eb2234ca9c82d/users/2cb56dbc47774ceeb9281cd329ffcc71/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:19.818088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b37f29e0cc94671847eb2234ca9c82d, user_id=2cb56dbc47774ceeb9281cd329ffcc71, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:19.839507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:19.911871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 897/1806] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:19 2022] PUT /identity/v3/projects/2b37f29e0cc94671847eb2234ca9c82d/users/2cb56dbc47774ceeb9281cd329ffcc71/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:19.923709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b23ed-9988-470c-8325-07231a5299ad None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:19.923993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b23ed-9988-470c-8325-07231a5299ad None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:19.924271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b23ed-9988-470c-8325-07231a5299ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:20.003398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-719b23ed-9988-470c-8325-07231a5299ad None None] MFA Rules not processed for user `2cb56dbc47774ceeb9281cd329ffcc71`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:20.004684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-719b23ed-9988-470c-8325-07231a5299ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:20.222895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:20.225672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 910/1807] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:20.451504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a682b-b4ef-48ec-a17a-0bf8003d73a4 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:20.490344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e19a11-af17-4c71-a644-ba4a9e28e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:20.491023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e19a11-af17-4c71-a644-ba4a9e28e8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:20.491300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e19a11-af17-4c71-a644-ba4a9e28e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:20.491604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e19a11-af17-4c71-a644-ba4a9e28e8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:20.492260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e19a11-af17-4c71-a644-ba4a9e28e8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:20.494227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e19a11-af17-4c71-a644-ba4a9e28e8d3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:20.580538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 898/1808] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:20 2022] POST /identity/v3/projects => generated 375 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:45:20.597429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b23ed-9988-470c-8325-07231a5299ad None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:20.620597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b23ed-9988-470c-8325-07231a5299ad None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:20.772682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:20.853924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:20.854574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:20.854856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:20.855139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:20.855832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:20.862690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:20.927194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea4247-af8f-42a6-83ea-591d2de0f284 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:20.928285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea4247-af8f-42a6-83ea-591d2de0f284 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:20.928775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea4247-af8f-42a6-83ea-591d2de0f284 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:20.931752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 911/1809] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:20 2022] POST /identity/v3/users => generated 294 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:20.932843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:20.934824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 899/1810] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:45:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:45:20.961969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da2e53-9a43-44c6-b01f-f23eb97c912a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:20.967035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:20.968011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:20.970305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:20.970580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:20.971612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:20.973662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:20.981003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:20.981340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:20.981838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:20.996956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 912/1811] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:21.001451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:21.018575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfa1ff-2c49-4eca-b318-975329661fdb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:21.024854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:21.025748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:21.026211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:21.026665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] PATH_INFO: `/v3/projects/d8cc5f380ddc4d4d9deeb25370f51059/users/d06cca25292f4419a70ad07b81cec8c8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:21.053821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:21.055323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:21.066312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cc5f380ddc4d4d9deeb25370f51059, user_id=d06cca25292f4419a70ad07b81cec8c8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:21.083378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:21.105189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 913/1812] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:21 2022] PUT /identity/v3/projects/d8cc5f380ddc4d4d9deeb25370f51059/users/d06cca25292f4419a70ad07b81cec8c8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:21.106001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:21.107153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:21.194122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 900/1813] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:45:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:21.203786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6f43a-479d-4f36-9bd3-eeadd271d9d0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:21.208869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:21.209489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:21.209778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:21.210049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:21.210697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:21.212987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:21.222011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:21.223656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:21.224524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 914/1814] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:21.229683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:21.246702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70801590-0c58-4ca1-a503-542a2965b9a6 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:21.248874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083c087-90a6-4c62-b48b-bc6f63193d8e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:21.256523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932bf6d-b757-41c6-a04a-f254155642a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:21.257303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932bf6d-b757-41c6-a04a-f254155642a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:21.257704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932bf6d-b757-41c6-a04a-f254155642a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:21.258518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932bf6d-b757-41c6-a04a-f254155642a6 admin admin] PATH_INFO: `/v3/projects/d8cc5f380ddc4d4d9deeb25370f51059/users/d06cca25292f4419a70ad07b81cec8c8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:21.261240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932bf6d-b757-41c6-a04a-f254155642a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cc5f380ddc4d4d9deeb25370f51059, user_id=d06cca25292f4419a70ad07b81cec8c8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:21.264140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932bf6d-b757-41c6-a04a-f254155642a6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:21.275914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 915/1815] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:21 2022] PUT /identity/v3/projects/d8cc5f380ddc4d4d9deeb25370f51059/users/d06cca25292f4419a70ad07b81cec8c8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:21.289450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:21.289450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:21.289834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:21.336561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] MFA Rules not processed for user `d06cca25292f4419a70ad07b81cec8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:21.338463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:21.398928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:21.399745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:21.400319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:21.400700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:21.413054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:21.414922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:21.457518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 901/1816] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:21.470547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:21.498342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 916/1817] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:21.512390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:21.513972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:21.616771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee680c9d-44dd-41a7-bc43-4141b6a1c437 service nova] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:21.686002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:21.687183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:21.688012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:21.688514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:21.746781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:21.861858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:21.864258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:21.880622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 902/1818] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:23.944087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:23.945548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:24.121325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e879f-1926-48c5-a0c7-f0d3d2efcc07 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:24.125331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:24.134744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a55e3-f202-4fa8-8938-9ec33d9a23c1 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:24.150392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:24.151626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:24.153398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:24.153602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0232f-bebb-4c7c-b8cd-293668a2b286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:24.154416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0232f-bebb-4c7c-b8cd-293668a2b286 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:24.154875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] PATH_INFO: `/v3/users/999fab069ac54929a957cfe640f5cbbe` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:24.156281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0232f-bebb-4c7c-b8cd-293668a2b286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:24.156829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0232f-bebb-4c7c-b8cd-293668a2b286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:24.161279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=999fab069ac54929a957cfe640f5cbbe)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:24.163241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:24.273430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999fab069ac54929a957cfe640f5cbbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:24.299382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999fab069ac54929a957cfe640f5cbbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:24.310571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0232f-bebb-4c7c-b8cd-293668a2b286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:24.312703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999fab069ac54929a957cfe640f5cbbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:24.314208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999fab069ac54929a957cfe640f5cbbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:24.314539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0232f-bebb-4c7c-b8cd-293668a2b286 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:24.335179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 903/1819] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:24.339549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999fab069ac54929a957cfe640f5cbbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:24.351349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 917/1820] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:23 2022] DELETE /identity/v3/users/999fab069ac54929a957cfe640f5cbbe => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:24.355909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:24.356236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:24.356592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:24.398388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:24.400104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:24.594571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 904/1821] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:24.610465 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:24.612348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:24.768784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c909b-d91c-4d33-80a2-93734efe54fe admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:24.772589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:24.773207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:24.773486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:24.773760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:24.784955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:24.786760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:24.797902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 918/1822] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:25.580176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:25.581636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:25.586709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:25.588464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:25.604564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365092-8ab8-4fda-ad70-106f632e0bc8 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:25.629786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:25.630403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:25.630579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:25.630807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] PATH_INFO: `/v3/projects/d75190e5db3e4078a5077ccd02e38329` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:25.632109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75190e5db3e4078a5077ccd02e38329)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:25.638711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:25.661096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e51c97-0b4b-495f-8d41-b7524645d95f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:25.665030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:25.665747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:25.666085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:25.666420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:25.668287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:25.668659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:25.682050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:25.725997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75190e5db3e4078a5077ccd02e38329', 'request_id': 'req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:25.743397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75190e5db3e4078a5077ccd02e38329', 'request_id': 'req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:25.753947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75190e5db3e4078a5077ccd02e38329', 'request_id': 'req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:25.772794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] The token cache is being invalidate because project d75190e5db3e4078a5077ccd02e38329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:25.773097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:25.773790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:25.786059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 919/1823] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:25 2022] DELETE /identity/v3/projects/d75190e5db3e4078a5077ccd02e38329 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:25.793572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:25.808555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:25.819324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:25.821068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:25.833680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 905/1824] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:25.955079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62f664-d0e2-4ccd-808c-d2a08e39e42a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:25.974818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:25.975522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:25.975785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:25.976057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] PATH_INFO: `/v3/users/3eb8f4e306314c8ba2ccd4812fdef519` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:25.978332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb8f4e306314c8ba2ccd4812fdef519)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:25.981539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:26.176618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb8f4e306314c8ba2ccd4812fdef519'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.223712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb8f4e306314c8ba2ccd4812fdef519'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.242497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb8f4e306314c8ba2ccd4812fdef519'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.243174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb8f4e306314c8ba2ccd4812fdef519'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.252706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb8f4e306314c8ba2ccd4812fdef519'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.260192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 920/1825] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:25 2022] DELETE /identity/v3/users/3eb8f4e306314c8ba2ccd4812fdef519 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:26.719006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:26.732395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7ee75-4e37-4fd0-9a18-2c51c962d447 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:26.840742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:26.842358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:26.849242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:26.850118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] PATH_INFO: `/v3/projects/c2749459fa9a4059a6bd3edf09ccad39` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:26.851752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2749459fa9a4059a6bd3edf09ccad39)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:26.853906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:26.935759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2749459fa9a4059a6bd3edf09ccad39', 'request_id': 'req-3b51d35c-5b94-41f6-abd7-9158189f86a1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.944687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2749459fa9a4059a6bd3edf09ccad39', 'request_id': 'req-3b51d35c-5b94-41f6-abd7-9158189f86a1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.950959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2749459fa9a4059a6bd3edf09ccad39', 'request_id': 'req-3b51d35c-5b94-41f6-abd7-9158189f86a1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.965895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] The token cache is being invalidate because project c2749459fa9a4059a6bd3edf09ccad39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:26.966185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.966850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:26.968710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 906/1826] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:26 2022] DELETE /identity/v3/projects/c2749459fa9a4059a6bd3edf09ccad39 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:26.972421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:26.973857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:26.992847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52478-5fcc-4beb-8b90-e894a6493180 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:27.010511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:27.056160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:27.059987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.060556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.060909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] PATH_INFO: `/v3/users/58e56f2c29a44fb988b83076b5a69e08` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:27.063138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e56f2c29a44fb988b83076b5a69e08)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:27.066288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:27.121612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e56f2c29a44fb988b83076b5a69e08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.137837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e56f2c29a44fb988b83076b5a69e08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.153928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e56f2c29a44fb988b83076b5a69e08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.153928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e56f2c29a44fb988b83076b5a69e08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.165566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7e327b3-aab3-4c05-ae64-1dff47e331b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e56f2c29a44fb988b83076b5a69e08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.173730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 921/1827] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:26 2022] DELETE /identity/v3/users/58e56f2c29a44fb988b83076b5a69e08 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:27.206072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:27.208306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:27.231617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51d35c-5b94-41f6-abd7-9158189f86a1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:27.277049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:27.335090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:27.335818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.336101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.336401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] PATH_INFO: `/v3/users/eb5035771789445990e1f672774b4d2f` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:27.342509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5035771789445990e1f672774b4d2f)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:27.345227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:27.463135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5035771789445990e1f672774b4d2f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.474200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5035771789445990e1f672774b4d2f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.474878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5035771789445990e1f672774b4d2f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.479306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5035771789445990e1f672774b4d2f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.484963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5035771789445990e1f672774b4d2f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.510032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 907/1828] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:27 2022] DELETE /identity/v3/users/eb5035771789445990e1f672774b4d2f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:27.516291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.516696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.517033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:27.540780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:27.542085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:27.571818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:27.574044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:27.646524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 922/1829] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:27.657904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:27.659603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:27.671679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4909a-4121-4c87-bf24-c752609faeaf admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:27.677858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:27.678750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.679100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.679457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] PATH_INFO: `/v3/projects/cafb82089d6c4ee5aefd8d8e0dc37cef` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:27.680706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cafb82089d6c4ee5aefd8d8e0dc37cef)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:27.682802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:27.722359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:27.750368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3ebf3-8edb-4fe5-a14d-d9dbf687732d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:27.771354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cafb82089d6c4ee5aefd8d8e0dc37cef', 'request_id': 'req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.783826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cafb82089d6c4ee5aefd8d8e0dc37cef', 'request_id': 'req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.787795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:27.795579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cafb82089d6c4ee5aefd8d8e0dc37cef', 'request_id': 'req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.814025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] The token cache is being invalidate because project cafb82089d6c4ee5aefd8d8e0dc37cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:27.814794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.816005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1b5d2c3-1d5d-4e9e-bd0a-30c9c2cf6178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:27.820456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 908/1830] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:27 2022] DELETE /identity/v3/projects/cafb82089d6c4ee5aefd8d8e0dc37cef => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:27.826654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e9b4-a5e6-4b1b-9cda-0a2e94d945f1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.827034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e9b4-a5e6-4b1b-9cda-0a2e94d945f1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.827578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e9b4-a5e6-4b1b-9cda-0a2e94d945f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:27.832221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:27.839686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.840202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.840674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:27.843463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:27.866456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a3c0e9b4-a5e6-4b1b-9cda-0a2e94d945f1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:27.867717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a3c0e9b4-a5e6-4b1b-9cda-0a2e94d945f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:27.980446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:27.981745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:27.987521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 909/1831] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:27.993962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:27.994357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:27.994748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.019771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 923/1832] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:28.041192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:28.043182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:28.057594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 910/1833] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:28.060645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.085037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dc870-2dee-4106-974f-fec93c98c3b2 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.090051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.090740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.091449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.091822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.092486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:28.093338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.095278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.110795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 924/1834] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:28.116051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.126484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd33d5-b7b9-45bc-9e7e-3aeeae89b5c2 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.132160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.132785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.133629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.133966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.134825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.137535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.168698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 911/1835] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:28 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:28.173387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.185420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff118-a1d1-4a8a-9338-4584fe7a5246 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.288987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.290368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.290368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.290575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.291069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.293535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.323158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 925/1836] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:28 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:28.326979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.342401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098387b1-2c84-492d-ae64-9791997d3212 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.348687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.349449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.350635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.350917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.351613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.353410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.370430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 912/1837] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:28.374581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.386353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b878-0384-442e-b471-119809ca4b12 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.393457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd3994-8c2f-45d9-b93a-dafa7fbf08a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.394183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd3994-8c2f-45d9-b93a-dafa7fbf08a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.394529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd3994-8c2f-45d9-b93a-dafa7fbf08a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.394914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd3994-8c2f-45d9-b93a-dafa7fbf08a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.396621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd3994-8c2f-45d9-b93a-dafa7fbf08a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.398396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd3994-8c2f-45d9-b93a-dafa7fbf08a5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.428237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 926/1838] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:28.430395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.444455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74cf07-83f8-48af-b33c-fb91f22c8679 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.453294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.453989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.454267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.454536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] PATH_INFO: `/v3/projects/a9869728dfc842ba943152380e3cd9de/users/4910e79936ac4810af101fb7fb349cee/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.477377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9869728dfc842ba943152380e3cd9de, user_id=4910e79936ac4810af101fb7fb349cee, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.482369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.508416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 913/1839] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:28 2022] PUT /identity/v3/projects/a9869728dfc842ba943152380e3cd9de/users/4910e79936ac4810af101fb7fb349cee/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:28.523991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.524347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.524548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.549615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:28.564581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] MFA Rules not processed for user `4910e79936ac4810af101fb7fb349cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:28.566709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:28.662705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.669285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 927/1840] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:28.831197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1d3f5-5f28-4f7c-b7ba-9e9801f43535 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:28.851097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:28.852402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:28.852952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:28.854328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:28.855744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:28.861981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:28.938201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 914/1841] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:28 2022] POST /identity/v3/projects => generated 385 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:28.944536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:28.970388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333830c-418f-4304-beb2-a3dee9173bcf None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:29.142073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:29.145632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:29.147943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:29.147943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:29.150468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:29.157533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:29.220625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 928/1842] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:28 2022] POST /identity/v3/users => generated 299 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:45:29.223461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:29.258504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5438d-68af-4174-ac8e-fe310982ba7b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:29.265127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:29.265844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:29.266059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:29.266334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:29.266994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:29.269322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:29.272238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:29.295673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:29.297926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 915/1843] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:29.326254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4332-0567-4784-b768-6fae528fb588 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:29.342323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:29.347683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:29.347970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:29.348188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] PATH_INFO: `/v3/projects/35fca22de6944360933dffe8ea3d534f/users/5440615eda53410e98202fed11b799ee/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:29.396835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fca22de6944360933dffe8ea3d534f, user_id=5440615eda53410e98202fed11b799ee, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:29.401208 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:29.423484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 929/1844] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:29 2022] PUT /identity/v3/projects/35fca22de6944360933dffe8ea3d534f/users/5440615eda53410e98202fed11b799ee/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:29.428901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:29.441747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbcbb9-4970-4247-869c-3d0cc51e5e9d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:29.515566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb4a20-1c86-4461-9115-60433225b25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:29.516153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb4a20-1c86-4461-9115-60433225b25c admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:29.516364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb4a20-1c86-4461-9115-60433225b25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:29.516642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb4a20-1c86-4461-9115-60433225b25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:29.517297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb4a20-1c86-4461-9115-60433225b25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:29.518977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb4a20-1c86-4461-9115-60433225b25c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:29.534404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 916/1845] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:29.536660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:29.555303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3b65-acb5-42d0-867d-9068cb9f2202 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:29.567301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:29.568331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:29.568584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:29.569113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] PATH_INFO: `/v3/projects/35fca22de6944360933dffe8ea3d534f/users/5440615eda53410e98202fed11b799ee/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:29.571877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fca22de6944360933dffe8ea3d534f, user_id=5440615eda53410e98202fed11b799ee, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:29.574464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:29.588692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 930/1846] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:29 2022] PUT /identity/v3/projects/35fca22de6944360933dffe8ea3d534f/users/5440615eda53410e98202fed11b799ee/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:29.600068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:29.600068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:29.600595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:29.647783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] MFA Rules not processed for user `5440615eda53410e98202fed11b799ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:29.649196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:29.850676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 917/1847] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:29.864216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:29.865968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:29.914956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:29.916541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:30.036293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07933f9b-4644-407f-93cc-c14c92dc2606 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:30.043963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0d06-c7ce-4740-a119-797276e463b7 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:30.052627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:30.053533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.053946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.056129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:30.057171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.057446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.057716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] PATH_INFO: `/v3/projects/e809723e7474499fb0e097d3d4f4e117` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.060226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e809723e7474499fb0e097d3d4f4e117)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:30.064000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:30.077481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.089639 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:30.126360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:30.183789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e809723e7474499fb0e097d3d4f4e117', 'request_id': 'req-7db105c7-9eaf-464b-85c6-9f2cdba16dba'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.195903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e809723e7474499fb0e097d3d4f4e117', 'request_id': 'req-7db105c7-9eaf-464b-85c6-9f2cdba16dba'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.207017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e809723e7474499fb0e097d3d4f4e117', 'request_id': 'req-7db105c7-9eaf-464b-85c6-9f2cdba16dba'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.213643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:30.215246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:30.226700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] The token cache is being invalidate because project e809723e7474499fb0e097d3d4f4e117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:30.227221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.228382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.234204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 918/1848] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:29 2022] DELETE /identity/v3/projects/e809723e7474499fb0e097d3d4f4e117 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:30.236270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:30.237349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 931/1849] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:30.237936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:30.334274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db105c7-9eaf-464b-85c6-9f2cdba16dba admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:30.341354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:30.342046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.342321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.342590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] PATH_INFO: `/v3/users/e0b0da2448684bf78ec14e502adca840` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.344157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b0da2448684bf78ec14e502adca840)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:30.346138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:30.490969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b0da2448684bf78ec14e502adca840'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.504842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b0da2448684bf78ec14e502adca840'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.505724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b0da2448684bf78ec14e502adca840'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.506426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b0da2448684bf78ec14e502adca840'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.518172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-78bd165d-4a1d-4655-bb42-207b4dad3548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b0da2448684bf78ec14e502adca840'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.552350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 919/1850] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:30 2022] DELETE /identity/v3/users/e0b0da2448684bf78ec14e502adca840 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:30.592140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:30.593838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:30.612193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe48df5-35ea-4710-bf43-be7826d28a72 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:30.614698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391279fc-0352-46ab-9975-f9d673df5a21 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.614989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391279fc-0352-46ab-9975-f9d673df5a21 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.615257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391279fc-0352-46ab-9975-f9d673df5a21 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.617287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 920/1851] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:45:30.629140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d218f24-a821-4964-8657-909535b5c810 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.629905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d218f24-a821-4964-8657-909535b5c810 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.630175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d218f24-a821-4964-8657-909535b5c810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.664715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5d218f24-a821-4964-8657-909535b5c810 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:30.666730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5d218f24-a821-4964-8657-909535b5c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:30.723076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:30.746779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:30.747560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.747845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.748118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] PATH_INFO: `/v3/users/29dc3c2da9064c0181a81accff957a13` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.756734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29dc3c2da9064c0181a81accff957a13)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:30.759391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:30.854669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 921/1852] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:45:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:30.874925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d218f24-a821-4964-8657-909535b5c810 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:30.878358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5d218f24-a821-4964-8657-909535b5c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:30.880895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29dc3c2da9064c0181a81accff957a13'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.897985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d218f24-a821-4964-8657-909535b5c810 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:30.912522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645742fb-2346-4cd7-9520-297e4e823283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:30.913217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645742fb-2346-4cd7-9520-297e4e823283 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.913499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645742fb-2346-4cd7-9520-297e4e823283 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.913772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645742fb-2346-4cd7-9520-297e4e823283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.925680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29dc3c2da9064c0181a81accff957a13'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.931418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645742fb-2346-4cd7-9520-297e4e823283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:30.933875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645742fb-2346-4cd7-9520-297e4e823283 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:30.934567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29dc3c2da9064c0181a81accff957a13'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.935333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29dc3c2da9064c0181a81accff957a13'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.945769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29dc3c2da9064c0181a81accff957a13'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:30.953473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 922/1853] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:30.954537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 932/1854] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:30 2022] DELETE /identity/v3/users/29dc3c2da9064c0181a81accff957a13 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:30.961003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:30.961358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:30.961867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:30.985409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:30.986502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:31.137714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 923/1855] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:31.165542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:31.167108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:31.212682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:31.312709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c21eeb-b6b4-481e-9b33-9e503d116560 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:31.318008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc3006-20e1-4643-b632-efd79ff2367d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:31.318709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc3006-20e1-4643-b632-efd79ff2367d service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:31.319073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc3006-20e1-4643-b632-efd79ff2367d service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:31.319352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc3006-20e1-4643-b632-efd79ff2367d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:31.330042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc3006-20e1-4643-b632-efd79ff2367d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:31.331565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc3006-20e1-4643-b632-efd79ff2367d service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:31.341972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 933/1856] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:31.903562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:31.905038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:31.919747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e56da5-b277-4ced-bc9b-7374a2834142 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:31.926994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:31.928235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:31.928716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:31.929133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] PATH_INFO: `/v3/projects/32002cc4096e4071ad3e61193ad31e02` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:31.930657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32002cc4096e4071ad3e61193ad31e02)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:31.932683 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:32.035302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32002cc4096e4071ad3e61193ad31e02', 'request_id': 'req-198ea81d-5525-404a-8097-bda7b805b7f0'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.062525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32002cc4096e4071ad3e61193ad31e02', 'request_id': 'req-198ea81d-5525-404a-8097-bda7b805b7f0'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.120063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32002cc4096e4071ad3e61193ad31e02', 'request_id': 'req-198ea81d-5525-404a-8097-bda7b805b7f0'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.148906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] The token cache is being invalidate because project 32002cc4096e4071ad3e61193ad31e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:32.149175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.151824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-198ea81d-5525-404a-8097-bda7b805b7f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.154958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 924/1857] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:31 2022] DELETE /identity/v3/projects/32002cc4096e4071ad3e61193ad31e02 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:32.163244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:32.163586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:32.165728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:32.207341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:32.208552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:32.357560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 934/1858] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:32.369618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:32.369952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:32.370339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:32.416792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:32.417876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:32.437850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 925/1859] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:32.442482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:32.457585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c139b-8955-43d1-939c-63329b56cf08 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:32.479739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:32.480654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:32.481099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:32.484105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:32.485452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:32.487100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:32.493240 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:32.553814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 935/1860] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:32.558378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:32.572761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458b5c8-d051-40f7-bdf1-6f7b7e8564d0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:32.601446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:32.602084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:32.602379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:32.602637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:32.603283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:32.605259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:32.695467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:32.697222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:32.717426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521d426-244e-43bd-8efc-4de8d1ac82dd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:32.732353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 926/1861] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:32 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:45:32.738518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:32.739503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:32.739734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:32.740221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:32.740706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] PATH_INFO: `/v3/projects/b16c7408b63043a094046a026dda098d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:32.745011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16c7408b63043a094046a026dda098d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:32.747786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:32.752399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:32.874085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b4cb6-0f2b-4331-a65f-64bf23648e84 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:32.883631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16c7408b63043a094046a026dda098d', 'request_id': 'req-161a5aaa-1456-4269-a847-ac3ba5207508'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.904353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16c7408b63043a094046a026dda098d', 'request_id': 'req-161a5aaa-1456-4269-a847-ac3ba5207508'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.919304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16c7408b63043a094046a026dda098d', 'request_id': 'req-161a5aaa-1456-4269-a847-ac3ba5207508'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.943621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] The token cache is being invalidate because project b16c7408b63043a094046a026dda098d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:32.944082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.944890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:32.946597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 936/1862] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:32 2022] DELETE /identity/v3/projects/b16c7408b63043a094046a026dda098d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:32.951043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:32.953463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:33.019967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:33.041655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:33.043916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.044728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.046035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.048876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:33.054170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:33.059487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a5aaa-1456-4269-a847-ac3ba5207508 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:33.072598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:33.073417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.073704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.073951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] PATH_INFO: `/v3/users/2cb56dbc47774ceeb9281cd329ffcc71` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.076071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb56dbc47774ceeb9281cd329ffcc71)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:33.080419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:33.107209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 927/1863] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:32 2022] POST /identity/v3/users => generated 294 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:33.112349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:33.115031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:33.133668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ea38d-c3b0-4fb1-98e9-aa95383b6dbb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:33.148833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:33.149815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.150264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.150707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.151827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:33.154522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:33.170158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 928/1864] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:33.173266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:33.189867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb56dbc47774ceeb9281cd329ffcc71'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:33.195409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a275946-563d-4f0f-96c7-039c5011881d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:33.210537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:33.211173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.211510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.211758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.212396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:33.214058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:33.228326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb56dbc47774ceeb9281cd329ffcc71'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:33.241584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb56dbc47774ceeb9281cd329ffcc71'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:33.242617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb56dbc47774ceeb9281cd329ffcc71'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:33.244376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 929/1865] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:33.250008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:33.254324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7c89929d-c3bd-4bbc-a34d-c60d809e4f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb56dbc47774ceeb9281cd329ffcc71'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:33.264960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98ab0e-0239-49bb-a87a-02506247bab6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:33.265690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 937/1866] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:32 2022] DELETE /identity/v3/users/2cb56dbc47774ceeb9281cd329ffcc71 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:33.363290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:33.364231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.364670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.365070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] PATH_INFO: `/v3/projects/1c21fa0dee254291a2818fa42b380a4d/users/0cf64112bc484d61bcfbc5c8da449f2c/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.383007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c21fa0dee254291a2818fa42b380a4d, user_id=0cf64112bc484d61bcfbc5c8da449f2c, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:33.386051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:33.399294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:33.408188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 930/1867] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:33 2022] PUT /identity/v3/projects/1c21fa0dee254291a2818fa42b380a4d/users/0cf64112bc484d61bcfbc5c8da449f2c/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:33.420293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24037-5558-4e25-b533-37432a589813 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.420909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24037-5558-4e25-b533-37432a589813 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.420909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24037-5558-4e25-b533-37432a589813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.504432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-53f24037-5558-4e25-b533-37432a589813 None None] MFA Rules not processed for user `0cf64112bc484d61bcfbc5c8da449f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:33.505617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-53f24037-5558-4e25-b533-37432a589813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:33.628033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 938/1868] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:33.632144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:33.634009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:33.768541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68abab-a051-4020-9407-c96a86025acb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:33.776273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:33.777512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:33.777884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:33.778346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:33.779204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:33.782534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:33.841355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 931/1869] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:33 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:33.845613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24037-5558-4e25-b533-37432a589813 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:33.867168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24037-5558-4e25-b533-37432a589813 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:34.002366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:34.040324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27160cd0-5fa0-4228-b68f-adebc48d2863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:34.041376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27160cd0-5fa0-4228-b68f-adebc48d2863 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.041894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27160cd0-5fa0-4228-b68f-adebc48d2863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.042325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27160cd0-5fa0-4228-b68f-adebc48d2863 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.043125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27160cd0-5fa0-4228-b68f-adebc48d2863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:34.049140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27160cd0-5fa0-4228-b68f-adebc48d2863 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:34.117888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 939/1870] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:33 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:34.120313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:34.140489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199ad38-2c36-4375-9f8c-4f1c50d9d35f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:34.152620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1195e-2306-4250-8a19-417d00c7d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:34.156223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1195e-2306-4250-8a19-417d00c7d1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.156828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1195e-2306-4250-8a19-417d00c7d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.157107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1195e-2306-4250-8a19-417d00c7d1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.157836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1195e-2306-4250-8a19-417d00c7d1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:34.165039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1195e-2306-4250-8a19-417d00c7d1b8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:34.188246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6ab15-d38d-426c-8f9a-34987865c8b5 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.188543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6ab15-d38d-426c-8f9a-34987865c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.188825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6ab15-d38d-426c-8f9a-34987865c8b5 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.191836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 940/1871] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:45:34.196426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 932/1872] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:34.198206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6ab15-d38d-426c-8f9a-34987865c8b5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:34.214522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.215473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.215473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.235406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6ab15-d38d-426c-8f9a-34987865c8b5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:34.263904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:34.263904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.263904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.263904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.264244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:34.271044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:34.287399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:34.293115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:34.312913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 941/1873] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:34.320875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:34.354820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d253ea-1d1c-453f-b23b-003c315d89d0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:34.371613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7059c-843c-4568-a664-32f350315234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:34.372544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7059c-843c-4568-a664-32f350315234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.374246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7059c-843c-4568-a664-32f350315234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.375110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7059c-843c-4568-a664-32f350315234 admin admin] PATH_INFO: `/v3/projects/62cfcd46f51946eba3781360247b11d0/users/57627452fb20420d97f551f6f5f9e836/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.427879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7059c-843c-4568-a664-32f350315234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62cfcd46f51946eba3781360247b11d0, user_id=57627452fb20420d97f551f6f5f9e836, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:34.430561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7059c-843c-4568-a664-32f350315234 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:34.480559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 942/1874] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:34 2022] PUT /identity/v3/projects/62cfcd46f51946eba3781360247b11d0/users/57627452fb20420d97f551f6f5f9e836/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:34.499359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.499359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.499359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.532835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 933/1875] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:45:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:34.582113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] MFA Rules not processed for user `57627452fb20420d97f551f6f5f9e836`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:34.583508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:34.694896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 943/1876] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:34.696276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:34.707716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25787-1aff-4264-902a-dbe38e855b9d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:34.765206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:34.767154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:34.851972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:34.853911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:34.854474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:34.854965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:34.856211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:34.861144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:34.886711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:34.950785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 934/1877] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:34 2022] POST /identity/v3/projects => generated 383 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:34.952374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:34.975068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca6b7f-6d62-4402-ae17-bbbfe7084746 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.039931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:35.047548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba3bdb-fd5a-4b0a-b26c-6e0e499fabf5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.054935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.055631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.055913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.056185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.056838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.059637 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.088913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 935/1878] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:34 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:35.093933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.109619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e058d1-f93e-43f3-8572-1ad6b0805865 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.113913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.114948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.115649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.115983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.116253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.116868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.117345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.117873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.118723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.120918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.129869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.132255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.135706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 936/1879] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:35.142781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.156988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987cc7f-c4e5-4958-a17c-5bf862c95c6a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.159811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 944/1880] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:35.176844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.178674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.180613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.181530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] PATH_INFO: `/v3/projects/55913c96612b4689aeefc61f3bb9032a/users/64095f3641be4f518a124d5d45cce497/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.224340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55913c96612b4689aeefc61f3bb9032a, user_id=64095f3641be4f518a124d5d45cce497, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.232720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.240189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.242290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:35.270508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8c859-45cf-4875-a4f9-bc70ac5a3c21 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.278025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 937/1881] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:35 2022] PUT /identity/v3/projects/55913c96612b4689aeefc61f3bb9032a/users/64095f3641be4f518a124d5d45cce497/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:35.284339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.374153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98968d-4d7b-4afb-af1b-e96a79e56e0e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.385426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.386088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.386362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.386630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] PATH_INFO: `/v3/projects/2b37f29e0cc94671847eb2234ca9c82d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.387785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b37f29e0cc94671847eb2234ca9c82d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.388578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.389429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.389843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.390025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.390643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.393235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.395054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.421379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 938/1882] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:35.428327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.448547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505d6a8-5b16-47d2-be56-75bebb2cd303 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.459998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b37f29e0cc94671847eb2234ca9c82d', 'request_id': 'req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.472855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b37f29e0cc94671847eb2234ca9c82d', 'request_id': 'req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.486041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b37f29e0cc94671847eb2234ca9c82d', 'request_id': 'req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.524238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] The token cache is being invalidate because project 2b37f29e0cc94671847eb2234ca9c82d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:35.524645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.526038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.528770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 945/1883] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:35 2022] DELETE /identity/v3/projects/2b37f29e0cc94671847eb2234ca9c82d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:35.533351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.534794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:35.537258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:35.571397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92536f8d-7c57-45c0-886a-228d0b126554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.572739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536f8d-7c57-45c0-886a-228d0b126554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.573094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536f8d-7c57-45c0-886a-228d0b126554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.573439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536f8d-7c57-45c0-886a-228d0b126554 admin admin] PATH_INFO: `/v3/projects/55913c96612b4689aeefc61f3bb9032a/users/64095f3641be4f518a124d5d45cce497/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.575713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92536f8d-7c57-45c0-886a-228d0b126554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55913c96612b4689aeefc61f3bb9032a, user_id=64095f3641be4f518a124d5d45cce497, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.579522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92536f8d-7c57-45c0-886a-228d0b126554 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.594067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 939/1884] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:35 2022] PUT /identity/v3/projects/55913c96612b4689aeefc61f3bb9032a/users/64095f3641be4f518a124d5d45cce497/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:35.608219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.609317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.609904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.617945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:35.664815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e8a1e-38b6-4cf4-80a5-a4cbdd5fd097 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:35.686755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] MFA Rules not processed for user `64095f3641be4f518a124d5d45cce497`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:35.688524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:35.772198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:35.774286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:35.774637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:35.775250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] PATH_INFO: `/v3/users/d06cca25292f4419a70ad07b81cec8c8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:35.786663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06cca25292f4419a70ad07b81cec8c8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:35.790543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:35.918447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06cca25292f4419a70ad07b81cec8c8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.926471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 940/1885] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:35.929235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.932545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:35.949143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06cca25292f4419a70ad07b81cec8c8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.964570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06cca25292f4419a70ad07b81cec8c8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.965883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06cca25292f4419a70ad07b81cec8c8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.981239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06cca25292f4419a70ad07b81cec8c8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:35.996675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 946/1886] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:35 2022] DELETE /identity/v3/users/d06cca25292f4419a70ad07b81cec8c8 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:35.996675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:35.998363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:36.046386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee34e-234d-44c1-95ae-6b89abbdd6ec None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:36.126570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:36.127312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:36.128067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:36.128933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:36.143581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:36.147943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e1d01-08cf-4e95-89a3-53d267030cfe admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:36.152021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:36.152679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:36.152960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:36.153233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:36.156442 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:36.207641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:36.211469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:36.276846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:36.279075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9892f-2a78-48a4-b93a-c18684059f82 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:36.285526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:36.288096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:36.300798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 941/1887] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:36.303614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 947/1888] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:36.726131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:36.727472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:36.883936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65104-84c5-4d0e-9c34-6d9495e8a83d service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:36.890588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:36.891299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:36.892770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:36.893125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] PATH_INFO: `/v3/projects/d8cc5f380ddc4d4d9deeb25370f51059` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:36.894259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cc5f380ddc4d4d9deeb25370f51059)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:36.896273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:36.997807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cc5f380ddc4d4d9deeb25370f51059', 'request_id': 'req-c88bf6ce-13b1-4762-897d-2ccb28b01147'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:37.015737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cc5f380ddc4d4d9deeb25370f51059', 'request_id': 'req-c88bf6ce-13b1-4762-897d-2ccb28b01147'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:37.034984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cc5f380ddc4d4d9deeb25370f51059', 'request_id': 'req-c88bf6ce-13b1-4762-897d-2ccb28b01147'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:37.068199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] The token cache is being invalidate because project d8cc5f380ddc4d4d9deeb25370f51059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:37.068734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:37.069428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:37.071245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 942/1889] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:36 2022] DELETE /identity/v3/projects/d8cc5f380ddc4d4d9deeb25370f51059 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:37.076590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e224acb-d7e0-49c5-9d6c-074875dd912b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.077731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e224acb-d7e0-49c5-9d6c-074875dd912b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.077731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e224acb-d7e0-49c5-9d6c-074875dd912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.117536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3e224acb-d7e0-49c5-9d6c-074875dd912b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:37.117536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3e224acb-d7e0-49c5-9d6c-074875dd912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:37.190170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.191768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:37.236975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 948/1890] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:45:37.263244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.267338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.268212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.301135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bf6ce-13b1-4762-897d-2ccb28b01147 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.306038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.306666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.307076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.307352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.309326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:37.315127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:37.318799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:37.330094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:37.349686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 949/1891] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:37.351173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.369486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e338758-f6d7-4ac1-ac2e-be6c8acfaa7c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.382525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.383156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.383454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.383715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.384368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:37.384667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:37.390302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:37.417261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 950/1892] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:37.424166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.439771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:37.441234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:37.447743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f871-5fa2-4717-92af-3511963626d8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.458487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 943/1893] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:37.459619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.461271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.462451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.464605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.465548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:37.469311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:37.525746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 951/1894] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:37 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:37.527098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.547411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e8f00-67fd-4d9f-8759-c833bf92e298 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.694118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.695055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.695507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.697347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.698490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:37.701950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:37.771467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 944/1895] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:37 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:37.772348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.802536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3bc1f-3b4d-4c0f-9031-d05929c31489 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.821132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.821770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.822049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.822318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.822968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:37.824617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:37.838624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 952/1896] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:37.844363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.864023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b6eda-4871-4d23-abf4-60137403e470 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.873033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a968f68-ebbc-4045-ac79-fd25e634480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.873897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a968f68-ebbc-4045-ac79-fd25e634480d admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.874245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a968f68-ebbc-4045-ac79-fd25e634480d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.875323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a968f68-ebbc-4045-ac79-fd25e634480d admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.876498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a968f68-ebbc-4045-ac79-fd25e634480d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:37.882350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a968f68-ebbc-4045-ac79-fd25e634480d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:37.904774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 945/1897] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:37.906148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:37.930123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff8a81-28d0-44c9-a40e-3f9a26726e60 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:37.948612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:37.949728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:37.950215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:37.950767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] PATH_INFO: `/v3/projects/9f933e8a94c54f5bbf3e891486c28ff9/users/4dd17272849c4d8b81be99319f55b9ee/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:37.990158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f933e8a94c54f5bbf3e891486c28ff9, user_id=4dd17272849c4d8b81be99319f55b9ee, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:38.012247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:38.046446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 953/1898] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:37 2022] PUT /identity/v3/projects/9f933e8a94c54f5bbf3e891486c28ff9/users/4dd17272849c4d8b81be99319f55b9ee/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:38.065275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:38.066114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:38.067679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:38.181965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] MFA Rules not processed for user `4dd17272849c4d8b81be99319f55b9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:38.183294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:38.401127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 946/1899] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:38.410904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:38.413477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:38.518553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:38.579840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba66b-9e28-43ba-a4cc-d1e6344dafdb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:38.586720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:38.587864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:38.588151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:38.588546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:38.613019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:38.614653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:38.625708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 954/1900] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:42.126079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:42.128351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:42.173069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:42.379102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8284eb-778b-42a3-b938-43668b599527 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:42.385242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:42.385880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:42.386223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:42.387078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] PATH_INFO: `/v3/users/4910e79936ac4810af101fb7fb349cee` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:42.387862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4910e79936ac4810af101fb7fb349cee)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:42.418548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:42.480357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:42.554128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4910e79936ac4810af101fb7fb349cee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:42.588757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4910e79936ac4810af101fb7fb349cee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:42.589672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4910e79936ac4810af101fb7fb349cee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:42.590416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4910e79936ac4810af101fb7fb349cee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:42.602135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-1477266a-21d9-48cd-be04-940ef0be728d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4910e79936ac4810af101fb7fb349cee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:42.645628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 947/1901] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:42 2022] DELETE /identity/v3/users/4910e79936ac4810af101fb7fb349cee => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:42.647581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:42.647873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:42.648148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:42.655650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e242-0a22-4a8f-ac70-ebfef9b00628 service glance] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:42.685364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:42.687625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:42.751662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:42.752338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:42.752814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:42.752923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:42.755642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:42.782273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 948/1902] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:42.796439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:42.798249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:42.839712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:42.841918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:42.853235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:42.859062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 955/1903] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:45:42.872613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2e8b5-27f4-42e7-8e33-6fae43fb60bb None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:42.882707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:42.883406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:42.884535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:42.885174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:42.905877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:42.907707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:42.927063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 949/1904] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:43.534339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:43.536198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:43.565141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250de7a9-d261-4896-b31d-0192bbc4f573 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:43.571771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:43.572869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:43.573471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:43.574064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] PATH_INFO: `/v3/projects/a9869728dfc842ba943152380e3cd9de` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:43.575553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9869728dfc842ba943152380e3cd9de)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:43.578216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:43.683463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:43.692093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9869728dfc842ba943152380e3cd9de', 'request_id': 'req-1e4767dd-c9c6-4700-942c-c944306cc0a7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:43.705172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9869728dfc842ba943152380e3cd9de', 'request_id': 'req-1e4767dd-c9c6-4700-942c-c944306cc0a7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:43.721104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9869728dfc842ba943152380e3cd9de', 'request_id': 'req-1e4767dd-c9c6-4700-942c-c944306cc0a7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:43.759112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] The token cache is being invalidate because project a9869728dfc842ba943152380e3cd9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:43.759513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:43.763487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:43.765352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 956/1905] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:43 2022] DELETE /identity/v3/projects/a9869728dfc842ba943152380e3cd9de => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:43.769527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:43.802662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba3116-41e3-4d4f-aa0e-afb9f22b2446 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:43.957307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:43.957818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:43.958227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:43.958762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] PATH_INFO: `/v3/users/5440615eda53410e98202fed11b799ee` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:43.961166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5440615eda53410e98202fed11b799ee)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:43.963312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:44.065638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5440615eda53410e98202fed11b799ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:44.084877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5440615eda53410e98202fed11b799ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:44.085664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5440615eda53410e98202fed11b799ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:44.086445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5440615eda53410e98202fed11b799ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:44.104867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-091e41ab-024b-4f23-9d5a-9bdf3495bb78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5440615eda53410e98202fed11b799ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:44.149178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 950/1906] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:43 2022] DELETE /identity/v3/users/5440615eda53410e98202fed11b799ee => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:45.012190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.014961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:45.035715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4767dd-c9c6-4700-942c-c944306cc0a7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.148540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.149322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.149632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.150033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] PATH_INFO: `/v3/projects/35fca22de6944360933dffe8ea3d534f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.151395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fca22de6944360933dffe8ea3d534f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.153593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.226556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fca22de6944360933dffe8ea3d534f', 'request_id': 'req-08a313af-dd6d-4661-8c6f-b1d980e1de1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.237517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fca22de6944360933dffe8ea3d534f', 'request_id': 'req-08a313af-dd6d-4661-8c6f-b1d980e1de1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.247152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fca22de6944360933dffe8ea3d534f', 'request_id': 'req-08a313af-dd6d-4661-8c6f-b1d980e1de1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.262174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] The token cache is being invalidate because project 35fca22de6944360933dffe8ea3d534f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:45.262511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.263312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08a313af-dd6d-4661-8c6f-b1d980e1de1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.267743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 957/1907] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:45 2022] DELETE /identity/v3/projects/35fca22de6944360933dffe8ea3d534f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:45.273274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.273845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.274461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.302324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:45.303670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:45.401926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 951/1908] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:45.403228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.403552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.403851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.431942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:45.444415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:45.461069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 958/1909] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:45.465449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.472695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:45.477306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df834bbb-f79f-45b7-ba99-7ba7b86ddbfb None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.483420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.484270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.484887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.485264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.486034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:45.486578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.489241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.515473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 952/1910] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:45:45.515658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.529847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ec447-d844-44b3-aaf8-910349335984 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.543008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.543008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.543008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.543008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.543402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.546190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.596467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 959/1911] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:45 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:45.602254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.617501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0bb60-bed7-4c44-b1d1-12316a67831d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.718661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.721519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.721519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.721519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.721519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.723172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.751971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 953/1912] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:45 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:45.760374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.760542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.762348 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:45.776816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d716d00-19a6-4739-8941-d5cf4bd93377 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.782628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4e5c1-cd9e-4956-bcf5-5c1f9665249f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.782900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.783613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.783895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.786672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] PATH_INFO: `/v3/users/0cf64112bc484d61bcfbc5c8da449f2c` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.788172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf64112bc484d61bcfbc5c8da449f2c)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.790418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.806627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.808636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.809107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.809890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.812230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.814304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.830708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 960/1913] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:45.834922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.861379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986f20-d5fc-4d47-82cf-f7f391b1992a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.865130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf64112bc484d61bcfbc5c8da449f2c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.867994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:45.868783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.876478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf64112bc484d61bcfbc5c8da449f2c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.877497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf64112bc484d61bcfbc5c8da449f2c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.878388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf64112bc484d61bcfbc5c8da449f2c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.881997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.882380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.883088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:45.885394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:45.892310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4d622f4-ef92-46b4-a905-1b7b4082b6ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf64112bc484d61bcfbc5c8da449f2c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:45.921320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 961/1914] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:45.924327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:45.928479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:45.935497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 954/1915] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:45 2022] DELETE /identity/v3/users/0cf64112bc484d61bcfbc5c8da449f2c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:45.938012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:45.938536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:45.939036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:45.946756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:45.964638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dadc3-70fe-420e-a3c4-c4102bab8e16 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:45.969761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:45.974693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:46.094035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:46.121040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 955/1916] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:46.136844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:46.141947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:46.151617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4a69b-4b71-44bc-9961-5e24aea69b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:46.152440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4a69b-4b71-44bc-9961-5e24aea69b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:46.152852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4a69b-4b71-44bc-9961-5e24aea69b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:46.153224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4a69b-4b71-44bc-9961-5e24aea69b7c admin admin] PATH_INFO: `/v3/projects/ef75ee4c38f14290a6c11fcbc7458318/users/07e2276bb3804576b97dbb9a16c44d08/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:46.174813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4a69b-4b71-44bc-9961-5e24aea69b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef75ee4c38f14290a6c11fcbc7458318, user_id=07e2276bb3804576b97dbb9a16c44d08, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:46.178537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4a69b-4b71-44bc-9961-5e24aea69b7c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:46.207713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 962/1917] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:45 2022] PUT /identity/v3/projects/ef75ee4c38f14290a6c11fcbc7458318/users/07e2276bb3804576b97dbb9a16c44d08/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:46.221089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:46.221426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:46.221683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:46.263328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] MFA Rules not processed for user `07e2276bb3804576b97dbb9a16c44d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:46.265620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:46.290897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5d446-7937-415d-bf2d-4cb266612e85 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:46.402856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 963/1918] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:46.417131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:46.418991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:46.439030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:46.439969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:46.443960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:46.444439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:46.507739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20cf2b-410d-4aa2-b9bd-89f8ac4e2d7d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:46.512595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:46.513097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:46.513418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:46.513721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:46.526176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:46.528004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:46.554749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 964/1919] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:46.635362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:46.637083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:46.665525 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 956/1920] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:47.375605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:47.378578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:47.399891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa218873-1aae-49fd-8824-32f96c5276ac service glance] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:47.406720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:47.407658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:47.408077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:47.408470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] PATH_INFO: `/v3/projects/1c21fa0dee254291a2818fa42b380a4d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:47.409881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c21fa0dee254291a2818fa42b380a4d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:47.412182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:47.514909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:47.617516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c21fa0dee254291a2818fa42b380a4d', 'request_id': 'req-35158a10-d16b-4ba2-936e-b73740b978fb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.628340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c21fa0dee254291a2818fa42b380a4d', 'request_id': 'req-35158a10-d16b-4ba2-936e-b73740b978fb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.639905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c21fa0dee254291a2818fa42b380a4d', 'request_id': 'req-35158a10-d16b-4ba2-936e-b73740b978fb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.656902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] The token cache is being invalidate because project 1c21fa0dee254291a2818fa42b380a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:47.657492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.658270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.663891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 965/1921] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:47 2022] DELETE /identity/v3/projects/1c21fa0dee254291a2818fa42b380a4d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:47.667743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:47.681411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e0774-b2b2-4a38-918e-8f2d1034a989 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:47.758849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:47.773006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:47.773674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:47.774267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:47.774539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] PATH_INFO: `/v3/users/57627452fb20420d97f551f6f5f9e836` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:47.776135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57627452fb20420d97f551f6f5f9e836)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:47.778085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:47.840253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57627452fb20420d97f551f6f5f9e836'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.850630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57627452fb20420d97f551f6f5f9e836'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.851360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57627452fb20420d97f551f6f5f9e836'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.851992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57627452fb20420d97f551f6f5f9e836'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.860923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57627452fb20420d97f551f6f5f9e836'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:47.890932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 957/1922] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:47 2022] DELETE /identity/v3/users/57627452fb20420d97f551f6f5f9e836 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:48.254815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:48.269573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:48.291525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35158a10-d16b-4ba2-936e-b73740b978fb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:48.374664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:48.377199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:48.377875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:48.377875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] PATH_INFO: `/v3/projects/62cfcd46f51946eba3781360247b11d0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:48.379301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=62cfcd46f51946eba3781360247b11d0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:48.381664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:48.445116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62cfcd46f51946eba3781360247b11d0', 'request_id': 'req-e7bed8df-c739-4f31-a833-5cfc4aca61cd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.454192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62cfcd46f51946eba3781360247b11d0', 'request_id': 'req-e7bed8df-c739-4f31-a833-5cfc4aca61cd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.461370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62cfcd46f51946eba3781360247b11d0', 'request_id': 'req-e7bed8df-c739-4f31-a833-5cfc4aca61cd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.473898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] The token cache is being invalidate because project 62cfcd46f51946eba3781360247b11d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:48.474163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.474826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.476487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 966/1923] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:48 2022] DELETE /identity/v3/projects/62cfcd46f51946eba3781360247b11d0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:48.480536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:48.482268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:48.497601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cf102-6728-48a7-9128-c8c6625fabf3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:48.564841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:48.565492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:48.565772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:48.566051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] PATH_INFO: `/v3/users/64095f3641be4f518a124d5d45cce497` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:48.567653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64095f3641be4f518a124d5d45cce497)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:48.570042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:48.632227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64095f3641be4f518a124d5d45cce497'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.643932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64095f3641be4f518a124d5d45cce497'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.645238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64095f3641be4f518a124d5d45cce497'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.645587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64095f3641be4f518a124d5d45cce497'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.679903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0a44f9e2-512e-4cff-ba17-869d66fd8c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64095f3641be4f518a124d5d45cce497'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:48.696633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:48.705992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 958/1924] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:48 2022] DELETE /identity/v3/users/64095f3641be4f518a124d5d45cce497 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:49.222847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.224360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:49.238304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bed8df-c739-4f31-a833-5cfc4aca61cd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:49.285401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:49.316816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:49.317564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.317906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.318188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] PATH_INFO: `/v3/projects/55913c96612b4689aeefc61f3bb9032a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.319313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=55913c96612b4689aeefc61f3bb9032a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:49.321258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:49.382773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55913c96612b4689aeefc61f3bb9032a', 'request_id': 'req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:49.393688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55913c96612b4689aeefc61f3bb9032a', 'request_id': 'req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:49.406195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55913c96612b4689aeefc61f3bb9032a', 'request_id': 'req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:49.431543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] The token cache is being invalidate because project 55913c96612b4689aeefc61f3bb9032a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:49.432019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:49.432804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d9a3e57-da1a-4910-acd1-1483fd2c2d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:49.440830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.441307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.441643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.446892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 967/1925] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:49 2022] DELETE /identity/v3/projects/55913c96612b4689aeefc61f3bb9032a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:49.468730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:49.470474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:49.555600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 959/1926] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:49.564764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22624acf-825b-487e-9376-372e09108ab4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.565043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22624acf-825b-487e-9376-372e09108ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.565483 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22624acf-825b-487e-9376-372e09108ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.588220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-22624acf-825b-487e-9376-372e09108ab4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:49.589903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-22624acf-825b-487e-9376-372e09108ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:49.606068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 968/1927] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:49.611256 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.621132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f364f04-9c0b-4047-a26a-8a3c78ebb65b None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:49.627470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:49.627826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.628108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.628379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.629037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:49.629339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:49.631045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:49.652786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 960/1928] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:49.657567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22624acf-825b-487e-9376-372e09108ab4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.667374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22624acf-825b-487e-9376-372e09108ab4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:49.673260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:49.673960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.674328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.674666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.675411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:49.677426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:49.719478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 969/1929] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:49 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:49.722983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.742001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586d99-8686-4ba5-98a3-78244e4b749f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:49.827059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:49.827059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.827059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.827059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.827568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:49.827568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:49.871926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 961/1930] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:49 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:49.875923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.887594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b77e7-90b1-40b0-8d73-a537c456f5c8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:49.899183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:49.899925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.900241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.900750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.901276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:49.904435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:49.920563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 970/1931] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:49.924284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.946771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc8375-77e0-46f7-8b82-79808d345dd2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:49.953561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a665b2f-ddc8-4d33-8b72-e1cdf638d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:49.954468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a665b2f-ddc8-4d33-8b72-e1cdf638d93b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:49.955097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a665b2f-ddc8-4d33-8b72-e1cdf638d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:49.955636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a665b2f-ddc8-4d33-8b72-e1cdf638d93b admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:49.956688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a665b2f-ddc8-4d33-8b72-e1cdf638d93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:49.959759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a665b2f-ddc8-4d33-8b72-e1cdf638d93b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:49.977157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 962/1932] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:49.981619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:49.999344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18b847-c1be-4215-8f80-6384428be127 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.005377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.006058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.006336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.006609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] PATH_INFO: `/v3/projects/2ac05b52047647de9d9199995e238ec5/users/bd7241e6fa2c43daa938bb4cd627b24f/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.024304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac05b52047647de9d9199995e238ec5, user_id=bd7241e6fa2c43daa938bb4cd627b24f, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.027102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.046321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 971/1933] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:49 2022] PUT /identity/v3/projects/2ac05b52047647de9d9199995e238ec5/users/bd7241e6fa2c43daa938bb4cd627b24f/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:50.065991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.066284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.066653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.116401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] MFA Rules not processed for user `bd7241e6fa2c43daa938bb4cd627b24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:50.118194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:50.221605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:50.248940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 963/1934] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:50.253510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:50.349644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:50.354988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda66d7-a46a-4a6d-836c-fcf9f5038b0b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.369270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.369906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.370183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.370469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.371098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.373085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.417727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 972/1935] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:50 2022] POST /identity/v3/projects => generated 401 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:50.426334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:50.447143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1240687-530a-45ee-a68f-6605b7b321e0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.542877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.543545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.543830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.544097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.544991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.547803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.577316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 964/1936] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:50 2022] POST /identity/v3/users => generated 307 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:50.578575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:50.597887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ba14f-df3a-49b0-aaed-c45067f96129 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.600981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.601765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.602115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.602459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.603201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.605035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.619750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 973/1937] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:50.624988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:50.637045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76363de3-9930-4938-a115-bb2a4ee32937 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.645397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.646372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.646805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.647237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] PATH_INFO: `/v3/projects/9aa0c6d3c8654f5c83d21ee61f0fcd88/users/761a2a36e3bd4a708fd074cb24f4ec99/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.669923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa0c6d3c8654f5c83d21ee61f0fcd88, user_id=761a2a36e3bd4a708fd074cb24f4ec99, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.673004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.689609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 965/1938] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:50 2022] PUT /identity/v3/projects/9aa0c6d3c8654f5c83d21ee61f0fcd88/users/761a2a36e3bd4a708fd074cb24f4ec99/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:50.693891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:50.708766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da90e-e0a4-47be-a7c1-eed30fb69295 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.787061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23126b43-06a0-4011-b2c0-d46717702a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.787697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23126b43-06a0-4011-b2c0-d46717702a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.787972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23126b43-06a0-4011-b2c0-d46717702a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.788238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23126b43-06a0-4011-b2c0-d46717702a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.788932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23126b43-06a0-4011-b2c0-d46717702a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.791361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23126b43-06a0-4011-b2c0-d46717702a55 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.807049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 974/1939] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:50.811482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:50.830721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44514e4d-a1a5-4c34-9882-0a5f1edbe869 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:50.845460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:50.845460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.845460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.845460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] PATH_INFO: `/v3/projects/9aa0c6d3c8654f5c83d21ee61f0fcd88/users/761a2a36e3bd4a708fd074cb24f4ec99/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.847254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa0c6d3c8654f5c83d21ee61f0fcd88, user_id=761a2a36e3bd4a708fd074cb24f4ec99, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:50.850694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:50.874200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 966/1940] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:50 2022] PUT /identity/v3/projects/9aa0c6d3c8654f5c83d21ee61f0fcd88/users/761a2a36e3bd4a708fd074cb24f4ec99/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:50.882791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:50.883086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:50.883363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:50.927461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] MFA Rules not processed for user `761a2a36e3bd4a708fd074cb24f4ec99`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:50.928558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:51.047377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 975/1941] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:50 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:51.054681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:51.058225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:51.120420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5e18f-6717-4fc2-b684-8db056cbca7e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:51.125003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:51.125630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:51.125910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:51.126183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:51.135786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:51.137257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:51.148239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 967/1942] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:51.237650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:51.249190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d2d5-20fc-4348-9b60-53d3b1d6622c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:51.253384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54d6a2-1165-4337-832a-b820f785a5e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:51.253993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d6a2-1165-4337-832a-b820f785a5e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:51.254268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d6a2-1165-4337-832a-b820f785a5e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:51.254533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d6a2-1165-4337-832a-b820f785a5e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:51.313807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:51.324165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d6a2-1165-4337-832a-b820f785a5e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:51.325611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d6a2-1165-4337-832a-b820f785a5e4 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:51.335288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 976/1943] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:55.741021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:55.789212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:55.809038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c632ed4-e920-4196-a559-c5e07fc3667a service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:55.814424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:55.815077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:55.815369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:55.815644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] PATH_INFO: `/v3/users/bd7241e6fa2c43daa938bb4cd627b24f` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:55.816905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7241e6fa2c43daa938bb4cd627b24f)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:55.818776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:55.874235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7241e6fa2c43daa938bb4cd627b24f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:55.882120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7241e6fa2c43daa938bb4cd627b24f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:55.883160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7241e6fa2c43daa938bb4cd627b24f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:55.884234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7241e6fa2c43daa938bb4cd627b24f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:55.893197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7241e6fa2c43daa938bb4cd627b24f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:55.911490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 968/1944] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:55 2022] DELETE /identity/v3/users/bd7241e6fa2c43daa938bb4cd627b24f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:55.915974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:55.916519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:55.916519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:55.936627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:55.937814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:55.997186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 977/1945] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:56.007874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:56.009401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:56.068542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74cd9-4ff3-4c1d-8ffa-18a941cdd7ac admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:56.072181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:56.072797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:56.073075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:56.073344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:56.082515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:56.083999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:56.093456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 969/1946] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:45:56.368185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:56.369703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:56.385483 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c00a80-df06-45e2-88c5-a1daca45f004 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:56.390241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:56.390880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:56.391162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:56.391459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] PATH_INFO: `/v3/projects/2ac05b52047647de9d9199995e238ec5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:56.392611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac05b52047647de9d9199995e238ec5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:56.394626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:56.451693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac05b52047647de9d9199995e238ec5', 'request_id': 'req-8af5e777-95e8-4e21-9b51-bce4633411a7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.472531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac05b52047647de9d9199995e238ec5', 'request_id': 'req-8af5e777-95e8-4e21-9b51-bce4633411a7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.479319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac05b52047647de9d9199995e238ec5', 'request_id': 'req-8af5e777-95e8-4e21-9b51-bce4633411a7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.490628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] The token cache is being invalidate because project 2ac05b52047647de9d9199995e238ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:56.490917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.491612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.493115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 978/1947] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:56 2022] DELETE /identity/v3/projects/2ac05b52047647de9d9199995e238ec5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:56.497129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:56.507586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487a6e0-ab2f-44ae-950c-4cf866367ecb service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:56.568077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:56.568718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:56.568997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:56.569292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] PATH_INFO: `/v3/users/761a2a36e3bd4a708fd074cb24f4ec99` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:56.570523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=761a2a36e3bd4a708fd074cb24f4ec99)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:56.572505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:56.583257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:56.629639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761a2a36e3bd4a708fd074cb24f4ec99'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.637994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761a2a36e3bd4a708fd074cb24f4ec99'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.638659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761a2a36e3bd4a708fd074cb24f4ec99'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.639409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761a2a36e3bd4a708fd074cb24f4ec99'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.646672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-89c53b89-dc0f-4e1d-8de1-f0d8bc51a30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761a2a36e3bd4a708fd074cb24f4ec99'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:56.664180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 970/1948] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:45:56 2022] DELETE /identity/v3/users/761a2a36e3bd4a708fd074cb24f4ec99 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:57.000313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.001824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:57.015403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5e777-95e8-4e21-9b51-bce4633411a7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.073764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.074398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.074671 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.074936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] PATH_INFO: `/v3/projects/9aa0c6d3c8654f5c83d21ee61f0fcd88` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.076054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa0c6d3c8654f5c83d21ee61f0fcd88)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.078035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.095077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:57.128014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa0c6d3c8654f5c83d21ee61f0fcd88', 'request_id': 'req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:57.135568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa0c6d3c8654f5c83d21ee61f0fcd88', 'request_id': 'req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:57.141365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa0c6d3c8654f5c83d21ee61f0fcd88', 'request_id': 'req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:57.152392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] The token cache is being invalidate because project 9aa0c6d3c8654f5c83d21ee61f0fcd88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:45:57.152680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:57.153318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5dbcf9d1-1ab1-4751-896e-1ed3646f6e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:45:57.155755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 979/1949] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:45:56 2022] DELETE /identity/v3/projects/9aa0c6d3c8654f5c83d21ee61f0fcd88 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:57.160671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.161127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.161230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.181982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:57.183205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:57.245132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 971/1950] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:57.253738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.254028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.254305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.274234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:57.275474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:57.284245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 980/1951] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:57.288469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.298986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f3357-3375-444d-9cbe-86039ecbb59e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.303703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.304298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.304572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.304865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.305494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:45:57.305816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.307441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.319401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 972/1952] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:45:57.323558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.333483 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1650-f7b8-4f8a-a735-3ffba56cd871 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.338172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.338772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.339051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.339321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.339976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.342506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.369953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 981/1953] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:57.374183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.384250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c56e6-49a6-432f-b67d-1f1184efb86d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.441926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.442444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.442718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.442988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.443652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.445552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.469349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 973/1954] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:57.473466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.484461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2c19-2ad7-4cf2-8a9f-c3c09fcf0b78 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.489154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.489761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.490043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.490315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.490954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.492624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.503177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 982/1955] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:57.507213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.526650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:57.526650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af01d89-bcb7-4b19-a59a-c9e0892fb57c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.526650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48cf64-aaec-4331-bde8-2d3a08fd7015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.526650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48cf64-aaec-4331-bde8-2d3a08fd7015 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.526970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48cf64-aaec-4331-bde8-2d3a08fd7015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.526970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48cf64-aaec-4331-bde8-2d3a08fd7015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.527657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48cf64-aaec-4331-bde8-2d3a08fd7015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.530217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48cf64-aaec-4331-bde8-2d3a08fd7015 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.542190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 974/1956] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:57.546750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.556877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaec4c-7a8f-498b-9eea-a786a6a08c61 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.561740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.562411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.562688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.562959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] PATH_INFO: `/v3/projects/e3a3960a1c8a4ba39df615adb987c47b/users/8eb2797762984508b798e16a4ccd68f1/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.578075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a3960a1c8a4ba39df615adb987c47b, user_id=8eb2797762984508b798e16a4ccd68f1, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.580773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.593792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 983/1957] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:57 2022] PUT /identity/v3/projects/e3a3960a1c8a4ba39df615adb987c47b/users/8eb2797762984508b798e16a4ccd68f1/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:57.605481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.605757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.606023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.634484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] MFA Rules not processed for user `8eb2797762984508b798e16a4ccd68f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:57.635701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:57.702821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 975/1958] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:57.707282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.768468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff3fb-6c9a-4f99-b72c-6fee1f020c06 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.773134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.773733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.774014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.774283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.774923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.776883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.808453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 984/1959] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:57.812636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.823301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60cf2a-e03a-4fd0-a596-785d1d6924e0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.878688 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.879292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.879605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.879879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.880503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.882431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.903417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 976/1960] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:57 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:57.908624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.919571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55a5ea-940e-4bde-a339-95c353fc8809 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.925519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.926122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.926396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.926690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.927327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.928946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.941537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 985/1961] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:45:57.945440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.956237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f3c37-33c7-4f14-95a5-2ab3884578e3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.961017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ed729-03b4-4f69-9ced-57eaed4d397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.961661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ed729-03b4-4f69-9ced-57eaed4d397a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.961967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ed729-03b4-4f69-9ced-57eaed4d397a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.962240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ed729-03b4-4f69-9ced-57eaed4d397a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:57.962923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ed729-03b4-4f69-9ced-57eaed4d397a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:57.964599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ed729-03b4-4f69-9ced-57eaed4d397a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:57.977094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 977/1962] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:57.981098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:57.992042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1f8fe-fccb-4dc2-ae57-b0767d1eb5f4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:57.996967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:57.997698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:57.998066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:57.998403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] PATH_INFO: `/v3/projects/95ef77fd86c24542a1a152c23913522b/users/79454c0711ef4b43a37a846737c3bb1a/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.019029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:58.023157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ef77fd86c24542a1a152c23913522b, user_id=79454c0711ef4b43a37a846737c3bb1a, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.026686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.043053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 986/1963] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:57 2022] PUT /identity/v3/projects/95ef77fd86c24542a1a152c23913522b/users/79454c0711ef4b43a37a846737c3bb1a/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:58.054212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.054505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.054779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.087135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] MFA Rules not processed for user `79454c0711ef4b43a37a846737c3bb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:58.088399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:58.141199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:58.163926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 978/1964] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:58.168431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.236232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f5f8-8916-4a41-874d-029a7c1a7fc8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.241436 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.242050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.242331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.242600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.243254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.245214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.283406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 987/1965] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:45:58 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:58.284697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.296578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823fc94-2f1a-4561-8936-bf1ef58d23ef None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.356982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.358424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.358830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.359101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.359771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.361736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.384225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 979/1966] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:45:58 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:45:58.388228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.398627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc04a4-e46a-4768-be85-4acca04c74b6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.403159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.403796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.404073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.404451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.405172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.406759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.417964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 988/1967] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:58.423192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.436548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d89f85-3df4-4cad-9e51-4a4726ac68a7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.441191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.441854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.442127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.442396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] PATH_INFO: `/v3/projects/f4e75ba7cd7946f9b0530a9e75ed2e1f/users/804d4397005f4fd99046175d1facd024/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.457248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e75ba7cd7946f9b0530a9e75ed2e1f, user_id=804d4397005f4fd99046175d1facd024, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.459961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.474201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 980/1968] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:58 2022] PUT /identity/v3/projects/f4e75ba7cd7946f9b0530a9e75ed2e1f/users/804d4397005f4fd99046175d1facd024/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:45:58.478290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.489275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cbe32-2a35-4ee0-a354-6d049b69fdc0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.550871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19730553-71af-455c-a7c3-6beb74cef4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.551518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19730553-71af-455c-a7c3-6beb74cef4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.551798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19730553-71af-455c-a7c3-6beb74cef4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.552067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19730553-71af-455c-a7c3-6beb74cef4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.552717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19730553-71af-455c-a7c3-6beb74cef4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.554347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19730553-71af-455c-a7c3-6beb74cef4b9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.564927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 989/1969] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:45:58.569102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.580277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10d099-3369-44f6-b7ea-56fe178054e2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.584957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.585618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.585930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.586169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] PATH_INFO: `/v3/projects/f4e75ba7cd7946f9b0530a9e75ed2e1f/users/804d4397005f4fd99046175d1facd024/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.588103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e75ba7cd7946f9b0530a9e75ed2e1f, user_id=804d4397005f4fd99046175d1facd024, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.590904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.605169 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 981/1970] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:45:58 2022] PUT /identity/v3/projects/f4e75ba7cd7946f9b0530a9e75ed2e1f/users/804d4397005f4fd99046175d1facd024/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:45:58.617728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.618011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.618282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.646764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] MFA Rules not processed for user `804d4397005f4fd99046175d1facd024`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:45:58.648006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:58.700511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:58.728092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 990/1971] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:45:58.739926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:45:58.741564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:45:58.801502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad521793-5424-4716-b8af-83ceb96ffc69 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:45:58.806389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:45:58.807007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:45:58.807281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:45:58.807581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:45:58.865317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:45:58.866808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:45:58.870630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:45:58.879168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 982/1972] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:45:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:00.420441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:00.432498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73f16b-de42-4b70-b859-068d60531a42 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:00.437644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:00.438515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:00.438927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:00.439326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:00.502020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:00.503473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:00.512847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 991/1973] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:01.222239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:01.230544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5e93-f6cf-41f1-b63f-aeed25241f49 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:01.234828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ab092-9c36-4dfd-8cda-6941d7336568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:01.235444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab092-9c36-4dfd-8cda-6941d7336568 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:01.235805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab092-9c36-4dfd-8cda-6941d7336568 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:01.236078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab092-9c36-4dfd-8cda-6941d7336568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:01.253931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ab092-9c36-4dfd-8cda-6941d7336568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:01.255392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ab092-9c36-4dfd-8cda-6941d7336568 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:01.265278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 983/1974] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:10.508771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=34 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:10.553876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:10.636444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb8b5-acb4-4feb-b400-c9b01d0eb056 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:10.641763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:10.642404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:10.642685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:10.642955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] PATH_INFO: `/v3/users/8eb2797762984508b798e16a4ccd68f1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:10.644121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb2797762984508b798e16a4ccd68f1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:10.646077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:10.697538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb2797762984508b798e16a4ccd68f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:10.709554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb2797762984508b798e16a4ccd68f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:10.725128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb2797762984508b798e16a4ccd68f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:10.725818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb2797762984508b798e16a4ccd68f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:10.733660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb2797762984508b798e16a4ccd68f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:10.739617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 992/1975] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:10 2022] DELETE /identity/v3/users/8eb2797762984508b798e16a4ccd68f1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:10.743992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:10.744490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:10.744584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:10.762918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:10.764135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:10.825875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 984/1976] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:10.836529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:10.837991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:10.907063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c1b25-a62b-41bb-a79c-dba1312b2b46 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:10.910772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:10.911388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:10.911695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:10.911965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:10.922177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:10.923643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:10.933915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 993/1977] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:11.214138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:11.215823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:11.229693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7886d59-923e-4edf-b76f-9684ad9818c8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:11.234407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:11.235090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:11.235361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:11.235664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] PATH_INFO: `/v3/projects/e3a3960a1c8a4ba39df615adb987c47b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:11.236789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a3960a1c8a4ba39df615adb987c47b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:11.238716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:11.293133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a3960a1c8a4ba39df615adb987c47b', 'request_id': 'req-e1602fc8-d779-479a-ba2b-3fdebc07b67a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.304773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a3960a1c8a4ba39df615adb987c47b', 'request_id': 'req-e1602fc8-d779-479a-ba2b-3fdebc07b67a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.311873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a3960a1c8a4ba39df615adb987c47b', 'request_id': 'req-e1602fc8-d779-479a-ba2b-3fdebc07b67a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.325106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] The token cache is being invalidate because project e3a3960a1c8a4ba39df615adb987c47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:11.326307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.326905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.328524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 985/1978] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:11 2022] DELETE /identity/v3/projects/e3a3960a1c8a4ba39df615adb987c47b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:11.332619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:11.342951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bfd47-bf74-4395-b37f-82e7bb134233 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:11.362045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:11.414290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:11.414962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:11.415249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:11.418696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] PATH_INFO: `/v3/users/79454c0711ef4b43a37a846737c3bb1a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:11.420562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79454c0711ef4b43a37a846737c3bb1a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:11.423013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:11.488893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79454c0711ef4b43a37a846737c3bb1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.505115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79454c0711ef4b43a37a846737c3bb1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.515283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79454c0711ef4b43a37a846737c3bb1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.515977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79454c0711ef4b43a37a846737c3bb1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.527178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79454c0711ef4b43a37a846737c3bb1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:11.534034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 994/1979] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:11 2022] DELETE /identity/v3/users/79454c0711ef4b43a37a846737c3bb1a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:11.848987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:11.848987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:11.863728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1602fc8-d779-479a-ba2b-3fdebc07b67a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:11.938635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:11.939273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:11.939608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:11.939988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] PATH_INFO: `/v3/projects/95ef77fd86c24542a1a152c23913522b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:11.941044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ef77fd86c24542a1a152c23913522b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:11.942960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:11.954236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:11.994376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ef77fd86c24542a1a152c23913522b', 'request_id': 'req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.003081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ef77fd86c24542a1a152c23913522b', 'request_id': 'req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.009834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ef77fd86c24542a1a152c23913522b', 'request_id': 'req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.021747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] The token cache is being invalidate because project 95ef77fd86c24542a1a152c23913522b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:12.022037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.022790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.024516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 986/1980] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:11 2022] DELETE /identity/v3/projects/95ef77fd86c24542a1a152c23913522b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:12.029189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:12.030699 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:12.043784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048cfef-8125-42c0-b44a-a8d33042d092 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:12.103992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:12.104630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:12.104910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:12.105175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] PATH_INFO: `/v3/users/804d4397005f4fd99046175d1facd024` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:12.106317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=804d4397005f4fd99046175d1facd024)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:12.108475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:12.166685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:12.169856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804d4397005f4fd99046175d1facd024'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.184783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804d4397005f4fd99046175d1facd024'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.194774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804d4397005f4fd99046175d1facd024'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.195782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804d4397005f4fd99046175d1facd024'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.206224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0b7049a7-e11f-4096-b930-8567ae4b84e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804d4397005f4fd99046175d1facd024'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.216518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 995/1981] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:12 2022] DELETE /identity/v3/users/804d4397005f4fd99046175d1facd024 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:12.513098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:12.514763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:12.550228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb1121-54aa-48be-98ee-dc1e8cbd7ac8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:12.627401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:12.628155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:12.628435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:12.628704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] PATH_INFO: `/v3/projects/f4e75ba7cd7946f9b0530a9e75ed2e1f` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:12.629859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e75ba7cd7946f9b0530a9e75ed2e1f)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:12.631874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:12.695756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e75ba7cd7946f9b0530a9e75ed2e1f', 'request_id': 'req-ca89d815-f997-46b1-8e6a-2a39b6142d9d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.706620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e75ba7cd7946f9b0530a9e75ed2e1f', 'request_id': 'req-ca89d815-f997-46b1-8e6a-2a39b6142d9d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.721112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e75ba7cd7946f9b0530a9e75ed2e1f', 'request_id': 'req-ca89d815-f997-46b1-8e6a-2a39b6142d9d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.739600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] The token cache is being invalidate because project f4e75ba7cd7946f9b0530a9e75ed2e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:12.739600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.739600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ca89d815-f997-46b1-8e6a-2a39b6142d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:12.739600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 987/1982] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:12 2022] DELETE /identity/v3/projects/f4e75ba7cd7946f9b0530a9e75ed2e1f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:12.743533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:12.744026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:12.744111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:12.768392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:12.769634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:12.843317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 996/1983] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:12.851280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:12.851806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:12.851914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:12.876265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:12.877568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:12.885856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 988/1984] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:12.890531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:12.901249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017207f-7236-4016-ba8b-befcc95ea4d8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:12.906895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:12.907529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:12.907805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:12.908072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:12.908719 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:12.909155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:12.910761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:12.923767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 997/1985] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:46:12.928250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:12.937894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3013e68-5efe-4eb9-90f9-7dc8cb446035 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:12.945531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:12.946305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:12.947681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:12.947681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:12.947681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:12.949443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:12.977378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:12.985904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 989/1986] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:12 2022] POST /identity/v3/projects => generated 397 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:12.990419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:13.001382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151f68a-86a2-4d73-b65a-4b08c6d04ff3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:13.086307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:13.086896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:13.087258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:13.087627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:13.088324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:13.090293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:13.125731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 998/1987] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:12 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:13.128037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:13.140548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c35a1-2ada-471a-8849-62aca5f62ce1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:13.145359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:13.146023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:13.146345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:13.146620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:13.147365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:13.148995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:13.163173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 990/1988] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:13.166574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:13.178295 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06169-bee4-4bc6-8fa9-404b40bb8b25 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:13.183043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6900f-3345-4105-96b8-b7988273d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:13.183681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6900f-3345-4105-96b8-b7988273d9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:13.183962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6900f-3345-4105-96b8-b7988273d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:13.184234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6900f-3345-4105-96b8-b7988273d9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:13.184951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6900f-3345-4105-96b8-b7988273d9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:13.187085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6900f-3345-4105-96b8-b7988273d9c9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:13.198858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 999/1989] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:13.204916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:13.215095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32a4bd-394a-49fc-9bb0-ab320a22c88a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:13.221808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:13.222470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:13.222745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:13.223013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] PATH_INFO: `/v3/projects/8fa5b2e421114e768caf8bc9bb6508fb/users/d90657c59d7c47b8b2a30aeb68bba7c7/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:13.238413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa5b2e421114e768caf8bc9bb6508fb, user_id=d90657c59d7c47b8b2a30aeb68bba7c7, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:13.241108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:13.256619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 991/1990] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:13 2022] PUT /identity/v3/projects/8fa5b2e421114e768caf8bc9bb6508fb/users/d90657c59d7c47b8b2a30aeb68bba7c7/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:13.267887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab0bb-524a-4053-bee6-72966d616be1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:13.268601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab0bb-524a-4053-bee6-72966d616be1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:13.268601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab0bb-524a-4053-bee6-72966d616be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:13.297776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-22fab0bb-524a-4053-bee6-72966d616be1 None None] MFA Rules not processed for user `d90657c59d7c47b8b2a30aeb68bba7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:13.298997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-22fab0bb-524a-4053-bee6-72966d616be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:13.312182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:13.373593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1000/1991] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:13.385777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:13.388733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:13.447572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ae1a-ccb4-4917-ae27-d23ce3a4f27b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:13.451785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b7b7a-fbcd-4351-a697-b350ea680dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:13.452386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b7b7a-fbcd-4351-a697-b350ea680dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:13.452660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b7b7a-fbcd-4351-a697-b350ea680dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:13.452926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b7b7a-fbcd-4351-a697-b350ea680dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:13.461583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b7b7a-fbcd-4351-a697-b350ea680dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:13.463018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b7b7a-fbcd-4351-a697-b350ea680dc5 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:13.473168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 992/1992] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:20.233496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace515-87ea-492e-a957-5e3b957583d4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:20.234502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace515-87ea-492e-a957-5e3b957583d4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:20.235331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace515-87ea-492e-a957-5e3b957583d4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:20.237306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1001/1993] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:46:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:46:20.250458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:20.251072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:20.251072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:20.282106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:20.283494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:20.355128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace515-87ea-492e-a957-5e3b957583d4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:20.358146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eaace515-87ea-492e-a957-5e3b957583d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:20.371143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 993/1994] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:46:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:20.382827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:20.384361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:20.397435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02dd69-3bca-44db-bc8c-38a996b4d907 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:20.402728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:20.403887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:20.404346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:20.404758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:20.413521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:20.415119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:20.428839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 994/1995] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:46:20.442498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace515-87ea-492e-a957-5e3b957583d4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:20.455876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:20.456528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:20.456813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:20.457091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] PATH_INFO: `/v3/users/89b20ec0ede5470eaa33e1ea934991d9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:20.458281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b20ec0ede5470eaa33e1ea934991d9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:20.460255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:20.525852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b20ec0ede5470eaa33e1ea934991d9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.540732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b20ec0ede5470eaa33e1ea934991d9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.551031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b20ec0ede5470eaa33e1ea934991d9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.552162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b20ec0ede5470eaa33e1ea934991d9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.562261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-365a25c3-1e99-4c94-b133-82aa875c1b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b20ec0ede5470eaa33e1ea934991d9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.570294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1002/1996] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:20 2022] DELETE /identity/v3/users/89b20ec0ede5470eaa33e1ea934991d9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:20.760540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:20.788513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40585-e055-48e0-b532-0c4577edd281 service placement] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:20.847805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:20.861870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:20.862972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:20.863351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:20.863808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] PATH_INFO: `/v3/projects/0900418261fc4370bbf30ffcc668c4c5` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:20.867944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0900418261fc4370bbf30ffcc668c4c5)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:20.868154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:20.929559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0900418261fc4370bbf30ffcc668c4c5', 'request_id': 'req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.936972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0900418261fc4370bbf30ffcc668c4c5', 'request_id': 'req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.942644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0900418261fc4370bbf30ffcc668c4c5', 'request_id': 'req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.954829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] The token cache is being invalidate because project 0900418261fc4370bbf30ffcc668c4c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:20.955195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.955888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f9249d0-dd4e-49ed-aa66-bb7f0101efe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:20.957373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 995/1997] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:20 2022] DELETE /identity/v3/projects/0900418261fc4370bbf30ffcc668c4c5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:20.962339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:20.962630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:20.962903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:20.982679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:20.983905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:21.013708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:21.043255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1003/1998] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:21.051756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472654f5-9c42-4609-9a49-60fda241774e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.052048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472654f5-9c42-4609-9a49-60fda241774e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.052324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472654f5-9c42-4609-9a49-60fda241774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.073461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-472654f5-9c42-4609-9a49-60fda241774e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:21.074648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-472654f5-9c42-4609-9a49-60fda241774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:21.083253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 996/1999] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:21.087505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.097703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3dab6-c3a0-421f-a2cb-8bae5a5de81f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.102750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.103323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.103615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.103884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.104531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:21.104829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.106452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.120514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1004/2000] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:46:21.122880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472654f5-9c42-4609-9a49-60fda241774e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.132468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472654f5-9c42-4609-9a49-60fda241774e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.137123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.138400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.138744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.139008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.139687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.141605 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.170206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 997/2001] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:21 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:21.174248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.184081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94c2e-475b-46be-ac0e-882b5a175085 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.240311 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.240948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.241176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.241446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.242094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.244040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.265942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1005/2002] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:21 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:21.270760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.286524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815fb2-f67a-478e-bd12-869d9a427811 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.291625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.292324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.292691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.293209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.293860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.299476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.311897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 998/2003] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:21.315938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.327708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df427-67c2-4f42-a5f2-fbf4a230a71c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.332507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a315c-4e53-4347-9c30-53e302630a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.333123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a315c-4e53-4347-9c30-53e302630a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.333765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a315c-4e53-4347-9c30-53e302630a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.334008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a315c-4e53-4347-9c30-53e302630a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.334980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a315c-4e53-4347-9c30-53e302630a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.337051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a315c-4e53-4347-9c30-53e302630a1b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.349028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1006/2004] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:21.353343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.369339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5947-f7e5-4d53-b8f8-9cc681102d4c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.377715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.378678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.378991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.379832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] PATH_INFO: `/v3/projects/c94e0d73da7c46a4b7b1a3f56a9d7204/users/176461fe42fb407a8388fde06f426dce/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.402306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94e0d73da7c46a4b7b1a3f56a9d7204, user_id=176461fe42fb407a8388fde06f426dce, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.408689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.443803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 999/2005] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:21 2022] PUT /identity/v3/projects/c94e0d73da7c46a4b7b1a3f56a9d7204/users/176461fe42fb407a8388fde06f426dce/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:21.465151 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.465475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.465842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.498432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] MFA Rules not processed for user `176461fe42fb407a8388fde06f426dce`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:21.500085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:21.580381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1007/2006] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:21.584900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.650617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481bd68-64f0-4ef4-a4ec-5b3aa0533cda admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.656554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.657187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.657477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.657744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.658396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.660358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.695210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1000/2007] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:21 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:21.699752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.711128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc235b8-f226-4025-b65c-caa65f1d84a2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.753686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:21.791011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.791724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.792085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.792493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.793161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.796405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.822757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1008/2008] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:21 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:21.827937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.833559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:21.841654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23400deb-e307-40c8-b653-7fd1850b5425 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.849475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.850532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.850532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.852396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.852846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.855143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.868601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1001/2009] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:21.872893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.883801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae7590-04fc-43bd-94ce-03e372aab3d1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:21.889528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:21.890103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:21.890412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:21.890679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] PATH_INFO: `/v3/projects/8d626ed0157246d18e3121e25b015b43/users/0e21e68f3bda4c309089ed5296feba42/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:21.908220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d626ed0157246d18e3121e25b015b43, user_id=0e21e68f3bda4c309089ed5296feba42, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:21.911147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:21.930688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1009/2010] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:21 2022] PUT /identity/v3/projects/8d626ed0157246d18e3121e25b015b43/users/0e21e68f3bda4c309089ed5296feba42/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:21.936330 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:21.953926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3e68b-77b4-4095-a38d-01fc61a360c0 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:22.023354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204dbcd-71fa-49e0-8c25-02c5a5fe0838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:22.023784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204dbcd-71fa-49e0-8c25-02c5a5fe0838 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:22.023942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204dbcd-71fa-49e0-8c25-02c5a5fe0838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:22.024244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204dbcd-71fa-49e0-8c25-02c5a5fe0838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:22.024852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204dbcd-71fa-49e0-8c25-02c5a5fe0838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:22.026449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204dbcd-71fa-49e0-8c25-02c5a5fe0838 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:22.041033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1002/2011] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:22.042372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:22.056518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c973d-bbe2-4566-8b2e-17a662b14194 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:22.061603 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:22.062390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:22.062601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:22.063579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] PATH_INFO: `/v3/projects/8d626ed0157246d18e3121e25b015b43/users/0e21e68f3bda4c309089ed5296feba42/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:22.069104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d626ed0157246d18e3121e25b015b43, user_id=0e21e68f3bda4c309089ed5296feba42, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:22.075159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:22.091670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1010/2012] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:22 2022] PUT /identity/v3/projects/8d626ed0157246d18e3121e25b015b43/users/0e21e68f3bda4c309089ed5296feba42/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:22.103951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:22.104258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:22.104510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:22.135755 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] MFA Rules not processed for user `0e21e68f3bda4c309089ed5296feba42`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:22.136992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:22.229849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1003/2013] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:22.242331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:22.244252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:22.308501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48121c6-6cf1-4897-b347-f3e32369fe10 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:22.313279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:22.313923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:22.314213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:22.314486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:22.324353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:22.325870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:22.336708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1011/2014] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:22.816018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:22.816713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:22.817042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:22.888367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888f4eb-ce16-488f-8708-18b8e6977c66 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:22.888705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2882-fd78-4e65-a5e6-796efab3a6cd None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:22.894957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:22.895624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:22.895898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:22.896165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] PATH_INFO: `/v3/users/d90657c59d7c47b8b2a30aeb68bba7c7` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:22.896453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:22.897087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:22.897365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:22.897457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90657c59d7c47b8b2a30aeb68bba7c7)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:22.897636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] PATH_INFO: `/v3/users/176461fe42fb407a8388fde06f426dce` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:22.898792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176461fe42fb407a8388fde06f426dce)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:22.899338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:22.900734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:22.915868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:22.934522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:22.959751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90657c59d7c47b8b2a30aeb68bba7c7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.961490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176461fe42fb407a8388fde06f426dce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.969533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90657c59d7c47b8b2a30aeb68bba7c7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.970427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90657c59d7c47b8b2a30aeb68bba7c7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.971115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90657c59d7c47b8b2a30aeb68bba7c7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.975769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176461fe42fb407a8388fde06f426dce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.980672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-13930dfc-63cd-4a4e-a233-787fc5dcc494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90657c59d7c47b8b2a30aeb68bba7c7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.984853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176461fe42fb407a8388fde06f426dce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.985497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176461fe42fb407a8388fde06f426dce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:22.994101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7f44a94e-8c5b-4974-81da-e9f78d6d50d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176461fe42fb407a8388fde06f426dce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.000842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1012/2015] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:22 2022] DELETE /identity/v3/users/176461fe42fb407a8388fde06f426dce => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:23.001682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1004/2016] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:22 2022] DELETE /identity/v3/users/d90657c59d7c47b8b2a30aeb68bba7c7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:23.006921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.007207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.007375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.007513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.007701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.008033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.027618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:23.027827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:23.029424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.029561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.093041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1013/2017] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:23.103069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.103648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1005/2018] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:23.104670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.113398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.118255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.167813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f2d89-8213-4a85-ba15-2c010f83cd93 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:23.172173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:23.172786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.173057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.173323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.182910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19c7b3-29a6-42eb-9a49-3258a810114c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:23.183147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:23.184456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:23.187086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:23.187829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.188110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.188380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.196975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1014/2019] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:46:23.201811 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:23.203270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:23.214067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1006/2020] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:46:23.584514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.586134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.604857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8778f-a9cd-48c3-8960-0bbce6470066 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:23.611121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:23.611812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.612087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.612424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] PATH_INFO: `/v3/projects/c94e0d73da7c46a4b7b1a3f56a9d7204` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.613531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94e0d73da7c46a4b7b1a3f56a9d7204)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:23.615473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:23.639030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.640658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.662609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e5e1-678f-45cd-bbf9-ee840497e112 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:23.670552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:23.671304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.671810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.672261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] PATH_INFO: `/v3/projects/8fa5b2e421114e768caf8bc9bb6508fb` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.675296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa5b2e421114e768caf8bc9bb6508fb)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:23.677335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:23.681833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:23.687756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94e0d73da7c46a4b7b1a3f56a9d7204', 'request_id': 'req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.701600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94e0d73da7c46a4b7b1a3f56a9d7204', 'request_id': 'req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.708550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94e0d73da7c46a4b7b1a3f56a9d7204', 'request_id': 'req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.722310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:23.722569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] The token cache is being invalidate because project c94e0d73da7c46a4b7b1a3f56a9d7204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:23.722753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.723457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.725137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1015/2021] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:23 2022] DELETE /identity/v3/projects/c94e0d73da7c46a4b7b1a3f56a9d7204 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:23.729183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.730581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.743725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa5b2e421114e768caf8bc9bb6508fb', 'request_id': 'req-8b7a0fac-5200-4720-874c-d0ce49882cc6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.754272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa5b2e421114e768caf8bc9bb6508fb', 'request_id': 'req-8b7a0fac-5200-4720-874c-d0ce49882cc6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.764977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa5b2e421114e768caf8bc9bb6508fb', 'request_id': 'req-8b7a0fac-5200-4720-874c-d0ce49882cc6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.782131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] The token cache is being invalidate because project 8fa5b2e421114e768caf8bc9bb6508fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:23.782477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.783158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b7a0fac-5200-4720-874c-d0ce49882cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.785128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1007/2022] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:23 2022] DELETE /identity/v3/projects/8fa5b2e421114e768caf8bc9bb6508fb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:23.791603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bcbb6-f2da-45a8-b83e-98e67f38b43d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.791953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bcbb6-f2da-45a8-b83e-98e67f38b43d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.792245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bcbb6-f2da-45a8-b83e-98e67f38b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.816040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e69bcbb6-f2da-45a8-b83e-98e67f38b43d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:23.817282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e69bcbb6-f2da-45a8-b83e-98e67f38b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.842569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132693-18e7-4491-bb96-ebf2b3ef1f4c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:23.891725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1008/2023] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:23.900995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.901307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.901589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.927081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:23.927081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.927081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.927081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] PATH_INFO: `/v3/users/0e21e68f3bda4c309089ed5296feba42` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.927760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e21e68f3bda4c309089ed5296feba42)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:23.929010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:23.929446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:23.930306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:23.938822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1009/2024] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:23.943864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.954607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41e01-d273-4010-abf6-bcd9bcf0d4d3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:23.958879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:23.959568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:23.959810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:23.960077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:23.960765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:23.961242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:23.963344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:23.977667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1010/2025] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:46:23.984101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:23.988359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e21e68f3bda4c309089ed5296feba42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:23.994446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135afc9a-ef05-4798-a513-0c6067bc973e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.000199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.000818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.001098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.001364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.001700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e21e68f3bda4c309089ed5296feba42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.002035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.003983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.014185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e21e68f3bda4c309089ed5296feba42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.014958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e21e68f3bda4c309089ed5296feba42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.025931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e21e68f3bda4c309089ed5296feba42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.028868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:24.034267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1016/2026] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:23 2022] DELETE /identity/v3/users/0e21e68f3bda4c309089ed5296feba42 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:24.037526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1011/2027] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:23 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:24.042463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.045219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:24.129668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef48022-69ce-4402-b129-e8aeff31c300 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.198811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.199460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.199756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.200449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.201118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.203105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.229887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1017/2028] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:24.238121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.253633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7187b-4360-41ff-bd8e-da4f9ec12ee1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.260890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.261531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.261822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.262094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.262949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.265246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.276898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1012/2029] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:24.281834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.297147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc52bbc-5110-4f08-b6a1-ee508eac38b5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.303182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.303831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.304108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.304378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.305024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.306632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.319081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1018/2030] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:24.323846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.334977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797b37-e76e-4634-b3d5-c428323c6765 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.337679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.339958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f0aa-71e4-4cc1-9eef-53bc49d55bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.340214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:24.341522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f0aa-71e4-4cc1-9eef-53bc49d55bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.341522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f0aa-71e4-4cc1-9eef-53bc49d55bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.341809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f0aa-71e4-4cc1-9eef-53bc49d55bd2 admin admin] PATH_INFO: `/v3/projects/8cb38d1aae344b868976df799b74b0f5/users/47773b38da48458fa356f34d259d3478/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.357247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a715b6-10b9-4fbb-bc1e-0c8494b57f81 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.361597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74f0aa-71e4-4cc1-9eef-53bc49d55bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb38d1aae344b868976df799b74b0f5, user_id=47773b38da48458fa356f34d259d3478, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.363718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.364374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.364658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.364933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] PATH_INFO: `/v3/projects/8d626ed0157246d18e3121e25b015b43` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.365774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74f0aa-71e4-4cc1-9eef-53bc49d55bd2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.366118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d626ed0157246d18e3121e25b015b43)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.368142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.383844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1013/2031] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:24 2022] PUT /identity/v3/projects/8cb38d1aae344b868976df799b74b0f5/users/47773b38da48458fa356f34d259d3478/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:24.389486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.389890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.390249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.411085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:24.412640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:24.424551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d626ed0157246d18e3121e25b015b43', 'request_id': 'req-9592e888-14a2-45a8-9112-9c365c5793d2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.433249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d626ed0157246d18e3121e25b015b43', 'request_id': 'req-9592e888-14a2-45a8-9112-9c365c5793d2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.443486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d626ed0157246d18e3121e25b015b43', 'request_id': 'req-9592e888-14a2-45a8-9112-9c365c5793d2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.456551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] The token cache is being invalidate because project 8d626ed0157246d18e3121e25b015b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:24.456837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.457532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9592e888-14a2-45a8-9112-9c365c5793d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:24.462961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1019/2032] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:24 2022] DELETE /identity/v3/projects/8d626ed0157246d18e3121e25b015b43 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:24.463188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:24.465316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36a1f-8fe3-4e0d-88d6-fc2e6873b4bc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.465316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36a1f-8fe3-4e0d-88d6-fc2e6873b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.465772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36a1f-8fe3-4e0d-88d6-fc2e6873b4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.483091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:24.490307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9be36a1f-8fe3-4e0d-88d6-fc2e6873b4bc None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:24.491636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1014/2033] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:24.498242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9be36a1f-8fe3-4e0d-88d6-fc2e6873b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:24.498445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.500841 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:24.508704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1020/2034] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:24.516796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.517332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.517568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.543400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:24.545662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:24.553974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1021/2035] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:24.558146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.563481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef1381-5bf9-4559-b341-5ec738d7aef9 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.567301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.567953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.568067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02933cc-9d74-4c9c-8619-8cecde01abd4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.568232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.568503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.574897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.574897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.574897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.575743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.576397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:24.576692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.579256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.580900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.582151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.595831 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1022/2036] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:46:24.596432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1015/2037] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:24.600399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.613953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c2731-7ef9-4e08-971a-237efcfa49fc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.620423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.621100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.621378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.621643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.622422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.624556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.659200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1023/2038] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:24.663933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.674645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164d9a-70ff-4c9a-a526-5dc94f5f2093 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.753947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.754588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.754858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.755126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.755837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.757987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.784206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1016/2039] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:46:24.789957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.819408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019d81-95e0-4744-83c8-f93cc2d04ef0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.827229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.828176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.828588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.828990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.830021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.832745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.851702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1024/2040] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:24.853400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.864721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca6182-b680-4035-959b-8f57134293e2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.870369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc35baa-001a-408a-bbb3-07ad81bfcc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.871068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc35baa-001a-408a-bbb3-07ad81bfcc73 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.871307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc35baa-001a-408a-bbb3-07ad81bfcc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.872703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc35baa-001a-408a-bbb3-07ad81bfcc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.873947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc35baa-001a-408a-bbb3-07ad81bfcc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.875665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc35baa-001a-408a-bbb3-07ad81bfcc73 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.887115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1017/2041] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:24.891951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:24.903948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe991ecc-a221-413d-b638-817935a24245 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:24.910623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:24.911441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.913496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.913496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] PATH_INFO: `/v3/projects/cb3d10f1ae354ffbaec670ea0c309f41/users/473330654baf435f848f3d7f24e38f44/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:24.940423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3d10f1ae354ffbaec670ea0c309f41, user_id=473330654baf435f848f3d7f24e38f44, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:24.943759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:24.969286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1025/2042] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:24 2022] PUT /identity/v3/projects/cb3d10f1ae354ffbaec670ea0c309f41/users/473330654baf435f848f3d7f24e38f44/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:24.984671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007d47-8552-4804-94b2-06585674f970 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:24.984950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007d47-8552-4804-94b2-06585674f970 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:24.985228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007d47-8552-4804-94b2-06585674f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.029624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b6007d47-8552-4804-94b2-06585674f970 None None] MFA Rules not processed for user `473330654baf435f848f3d7f24e38f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:25.031211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b6007d47-8552-4804-94b2-06585674f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:25.116312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:25.135035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1018/2043] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:25.140480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.229522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca93b5-45a3-4dff-aa37-02cbd5bea9f3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.235491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.236116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.236392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.236648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.237299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.240293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.280008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1026/2044] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:25 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:46:25.284814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007d47-8552-4804-94b2-06585674f970 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.308375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007d47-8552-4804-94b2-06585674f970 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.401131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.401979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.402400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.402701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.404046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.407058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.437504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1019/2045] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:25 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:25.439299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.451189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f383f5-7e09-4352-ae09-5a55145c7bc7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.458623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.459721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.459721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.460018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.460578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.462730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.479939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1027/2046] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:25.480920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.492875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dc492-52b8-41cf-bedd-7ecc97c51c05 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.500311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.501152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.501452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.501870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] PATH_INFO: `/v3/projects/8d0d4d4ccdcb4df7b3db58729e6e4575/users/b1772a0c64684e17ba19ec4e5ac12234/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.522353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0d4d4ccdcb4df7b3db58729e6e4575, user_id=b1772a0c64684e17ba19ec4e5ac12234, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.525602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.546625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1020/2047] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:25 2022] PUT /identity/v3/projects/8d0d4d4ccdcb4df7b3db58729e6e4575/users/b1772a0c64684e17ba19ec4e5ac12234/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:25.550855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.564993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ddb77-ac12-4907-aa71-7e9eac6f9012 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.641188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:25.649611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ddce6-afda-42d8-b9d3-581ad5aaa93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.650478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ddce6-afda-42d8-b9d3-581ad5aaa93b admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.651184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ddce6-afda-42d8-b9d3-581ad5aaa93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.651709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ddce6-afda-42d8-b9d3-581ad5aaa93b admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.652564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ddce6-afda-42d8-b9d3-581ad5aaa93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.655001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ddce6-afda-42d8-b9d3-581ad5aaa93b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.665876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1028/2048] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:25.671539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.683303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1052ac-463e-4f32-9d26-9bbc27b581e0 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.689957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.690667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.690953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.691230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] PATH_INFO: `/v3/projects/8d0d4d4ccdcb4df7b3db58729e6e4575/users/b1772a0c64684e17ba19ec4e5ac12234/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.693467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0d4d4ccdcb4df7b3db58729e6e4575, user_id=b1772a0c64684e17ba19ec4e5ac12234, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.696556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.710987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1021/2049] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:25 2022] PUT /identity/v3/projects/8d0d4d4ccdcb4df7b3db58729e6e4575/users/b1772a0c64684e17ba19ec4e5ac12234/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:25.721865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.722216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.722467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.757218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] MFA Rules not processed for user `b1772a0c64684e17ba19ec4e5ac12234`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:25.759374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:25.785821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.787724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:25.868762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1029/2050] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:25.882383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:25.885808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:25.930351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e37916-21cc-46fa-9def-04a2e31cec3c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.938457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.939267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.939696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.940067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] PATH_INFO: `/v3/users/4dd17272849c4d8b81be99319f55b9ee` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.942000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd17272849c4d8b81be99319f55b9ee)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:25.943953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:25.969153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3e79a-884f-49ea-a620-b9210a81f10c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:25.979457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce948d-3f05-4173-8ace-25057a9aa0ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:25.980135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce948d-3f05-4173-8ace-25057a9aa0ab service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:25.980599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce948d-3f05-4173-8ace-25057a9aa0ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:25.981862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce948d-3f05-4173-8ace-25057a9aa0ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:25.999610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce948d-3f05-4173-8ace-25057a9aa0ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:26.001804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce948d-3f05-4173-8ace-25057a9aa0ab service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:26.016677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd17272849c4d8b81be99319f55b9ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:26.019530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:26.032221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd17272849c4d8b81be99319f55b9ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:26.034338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd17272849c4d8b81be99319f55b9ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:26.035244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd17272849c4d8b81be99319f55b9ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:26.047418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd17272849c4d8b81be99319f55b9ee'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:26.053553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1030/2051] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:46:26.094891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1022/2052] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:25 2022] DELETE /identity/v3/users/4dd17272849c4d8b81be99319f55b9ee => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:26.100245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:26.100799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:26.108342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:26.141497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:26.142958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:26.161412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:26.163068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:26.295709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1031/2053] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:26.298910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ef634-78d3-4a0b-8d7b-0df46af0a263 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:26.305483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:26.305722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:26.306131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:26.306485 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:26.306966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:26.310536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:26.310819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:26.357912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:26.401308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b1608-a678-49ac-ac95-c97c76def223 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:26.406373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad02212-e837-49b1-aa15-4f1218bca35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:26.407164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad02212-e837-49b1-aa15-4f1218bca35b service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:26.407599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad02212-e837-49b1-aa15-4f1218bca35b service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:26.408041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad02212-e837-49b1-aa15-4f1218bca35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:26.442648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad02212-e837-49b1-aa15-4f1218bca35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:26.445951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad02212-e837-49b1-aa15-4f1218bca35b service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:26.457404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:26.459018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:26.471958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1023/2054] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:46:26.481924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1032/2055] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:26.971596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:26.973788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:26.996762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2bdc-0a6a-432a-9d4f-f1917a6dcb9e service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.010499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.011181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.011492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.011795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] PATH_INFO: `/v3/projects/9f933e8a94c54f5bbf3e891486c28ff9` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.013340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f933e8a94c54f5bbf3e891486c28ff9)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.015368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.077087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f933e8a94c54f5bbf3e891486c28ff9', 'request_id': 'req-82964e26-eff3-48d8-baf6-6bc193c85236'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:27.088840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f933e8a94c54f5bbf3e891486c28ff9', 'request_id': 'req-82964e26-eff3-48d8-baf6-6bc193c85236'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:27.098386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f933e8a94c54f5bbf3e891486c28ff9', 'request_id': 'req-82964e26-eff3-48d8-baf6-6bc193c85236'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:27.112455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] The token cache is being invalidate because project 9f933e8a94c54f5bbf3e891486c28ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:27.112763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:27.113597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-82964e26-eff3-48d8-baf6-6bc193c85236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:27.115261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1024/2056] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:26 2022] DELETE /identity/v3/projects/9f933e8a94c54f5bbf3e891486c28ff9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:27.122951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.122951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.122951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.144174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:27.145652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:27.232399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1033/2057] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:27.240879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.240879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.241195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.252889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:27.276001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:27.277810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:27.296583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1025/2058] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:27.297895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:27.311660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ae8b-1247-427d-a43c-162e0c267c72 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.320004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.320004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.320004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.320004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.320677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:27.322206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.322841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.338867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1034/2059] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:46:27.343984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:27.357156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46ede-efc9-4be8-9a27-97b8e4da6839 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.371653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.372738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.373243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.373511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.375310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.378217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.425735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1026/2060] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:27 2022] POST /identity/v3/projects => generated 389 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:27.429766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:27.440497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633e53-008f-4f83-85ed-6223a34c0b18 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.536258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.536909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.537191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.537463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.538123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.540294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.570786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1035/2061] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:27 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:27.573652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:27.593783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e8182-960b-4851-877d-5be353999f95 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.599652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.600390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.600769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.601145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.601943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.603707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.618640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1027/2062] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:27.620335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:27.631346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2b9bd-c709-4e00-b5b0-57374ac18b4d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.636540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e97625-97e4-4e12-970a-7ce4921c98c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.637303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97625-97e4-4e12-970a-7ce4921c98c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.638614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97625-97e4-4e12-970a-7ce4921c98c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.638962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97625-97e4-4e12-970a-7ce4921c98c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.639785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e97625-97e4-4e12-970a-7ce4921c98c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.641549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e97625-97e4-4e12-970a-7ce4921c98c7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.658577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1036/2063] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:27.660129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:27.671977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0402b7-8dfd-4b78-97ff-bd9ae1c7bd7a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:27.677680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:27.678304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.678782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.679680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] PATH_INFO: `/v3/projects/92d59ea2b397495eb68a79cf433e21c7/users/57b529553a4c44248d7323a1113f1b4d/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.717824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d59ea2b397495eb68a79cf433e21c7, user_id=57b529553a4c44248d7323a1113f1b4d, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:27.722969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:27.749924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1028/2064] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:27 2022] PUT /identity/v3/projects/92d59ea2b397495eb68a79cf433e21c7/users/57b529553a4c44248d7323a1113f1b4d/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:27.760038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:27.760683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:27.761444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:27.826726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] MFA Rules not processed for user `57b529553a4c44248d7323a1113f1b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:27.828222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:27.881859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:28.017777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1037/2065] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:28.056568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:28.058541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:28.187912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b6037-09d6-4c09-9ee6-87f68be77093 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:28.199783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abea8a-cbca-427a-9561-a379ab68146e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:28.200852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abea8a-cbca-427a-9561-a379ab68146e service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:28.201364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abea8a-cbca-427a-9561-a379ab68146e service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:28.201939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abea8a-cbca-427a-9561-a379ab68146e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:28.220315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abea8a-cbca-427a-9561-a379ab68146e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:28.227828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abea8a-cbca-427a-9561-a379ab68146e service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:28.263619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1029/2066] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:46:30.102363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.104345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:30.157290 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.157767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.157866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.221687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] MFA Rules not processed for user `47773b38da48458fa356f34d259d3478`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:30.222864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:30.252369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d545c12-5586-4f5d-946d-fc7a06e085fd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.259538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.260698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.261161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.261620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] PATH_INFO: `/v3/users/473330654baf435f848f3d7f24e38f44` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.263844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=473330654baf435f848f3d7f24e38f44)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.266326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.370762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1030/2067] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:30.373920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473330654baf435f848f3d7f24e38f44'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:30.375294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.377336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:30.394233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0ed5c-bd9d-4a98-88c5-eb881fc2eaa0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.394471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473330654baf435f848f3d7f24e38f44'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:30.399859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.400477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.400751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.401021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.401713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.404565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.406353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473330654baf435f848f3d7f24e38f44'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:30.409737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473330654baf435f848f3d7f24e38f44'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:30.420252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-eed92158-b4b0-4f6a-b1a3-6bbf91053f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473330654baf435f848f3d7f24e38f44'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:30.429910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1038/2068] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:30 2022] DELETE /identity/v3/users/473330654baf435f848f3d7f24e38f44 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:30.432096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18254-4035-4e00-a29c-af95023db11a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.432096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18254-4035-4e00-a29c-af95023db11a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.432450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18254-4035-4e00-a29c-af95023db11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.449532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1031/2069] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:30 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:46:30.451165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.453936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:30.456421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-35f18254-4035-4e00-a29c-af95023db11a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:30.472280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-35f18254-4035-4e00-a29c-af95023db11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:30.480428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:30.538651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:30.567107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bbbe8-0d93-4b83-a44b-a947b11ef7aa admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.568990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1039/2070] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:30.586846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f18254-4035-4e00-a29c-af95023db11a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.588507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-35f18254-4035-4e00-a29c-af95023db11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:30.590291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.591086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.591362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.591714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.592369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.594430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.621444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1032/2071] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:30 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:30.626014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.636455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22413c48-97c6-40a0-92fd-708bb2e7c903 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.642489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.643106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.643382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.643718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.644370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.645999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.658143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1033/2072] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:30.662163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.675995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd608-96be-45fb-9583-98afe9f10c6e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.676205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f18254-4035-4e00-a29c-af95023db11a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.680459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.680870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.681227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.681462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.681569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.681796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.681896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.682070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] PATH_INFO: `/v3/projects/4abf7cd541ca4a49961d89fe4ae24071/users/b590de08f34d4265a78860006a804f1c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.692445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.693926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.698951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abf7cd541ca4a49961d89fe4ae24071, user_id=b590de08f34d4265a78860006a804f1c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.702701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.708520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1040/2073] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:30.720044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1034/2074] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:30 2022] PUT /identity/v3/projects/4abf7cd541ca4a49961d89fe4ae24071/users/b590de08f34d4265a78860006a804f1c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:30.724977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.738883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5673-b0e6-43d7-9bf2-7278dc352d43 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.819148 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.820000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.820434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.820765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.821623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.823784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.837449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1041/2075] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:30.843138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:30.856615 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269fb0-37ca-4665-88b4-bd2c13ec7d08 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:30.866069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:30.866763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:30.867283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:30.867283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] PATH_INFO: `/v3/projects/4abf7cd541ca4a49961d89fe4ae24071/users/b590de08f34d4265a78860006a804f1c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:30.869341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abf7cd541ca4a49961d89fe4ae24071, user_id=b590de08f34d4265a78860006a804f1c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:30.872696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:30.885954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1035/2076] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:30 2022] PUT /identity/v3/projects/4abf7cd541ca4a49961d89fe4ae24071/users/b590de08f34d4265a78860006a804f1c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:31.057929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:31.060367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:31.094854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b6406-5000-4c40-a7b2-e6e18f8379d9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:31.198820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:31.198820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:31.198820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:31.198820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] PATH_INFO: `/v3/projects/cb3d10f1ae354ffbaec670ea0c309f41` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:31.200172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3d10f1ae354ffbaec670ea0c309f41)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:31.206618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:31.244105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:31.297294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3d10f1ae354ffbaec670ea0c309f41', 'request_id': 'req-bbd0df59-ca29-4371-861f-fc259e2da6c0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.314368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3d10f1ae354ffbaec670ea0c309f41', 'request_id': 'req-bbd0df59-ca29-4371-861f-fc259e2da6c0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.342365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3d10f1ae354ffbaec670ea0c309f41', 'request_id': 'req-bbd0df59-ca29-4371-861f-fc259e2da6c0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.363227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] The token cache is being invalidate because project cb3d10f1ae354ffbaec670ea0c309f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:31.365028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.365028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.366538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1042/2077] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:31 2022] DELETE /identity/v3/projects/cb3d10f1ae354ffbaec670ea0c309f41 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:31.371172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:31.374835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:31.413443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d857c-6f3b-4285-bd29-887ea7b08962 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:31.573839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:31.574548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:31.574833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:31.575398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] PATH_INFO: `/v3/users/b1772a0c64684e17ba19ec4e5ac12234` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:31.581500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1772a0c64684e17ba19ec4e5ac12234)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:31.583576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:31.680765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1772a0c64684e17ba19ec4e5ac12234'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.688400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:31.697849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1772a0c64684e17ba19ec4e5ac12234'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.700890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1772a0c64684e17ba19ec4e5ac12234'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.702761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1772a0c64684e17ba19ec4e5ac12234'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.717339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ece51e2-b45f-4c5a-bb53-1bb96c38d585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1772a0c64684e17ba19ec4e5ac12234'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:31.750736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1036/2078] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:31 2022] DELETE /identity/v3/users/b1772a0c64684e17ba19ec4e5ac12234 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:32.177201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.191851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0df59-ca29-4371-861f-fc259e2da6c0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.281337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.282078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.282740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.282740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] PATH_INFO: `/v3/projects/8d0d4d4ccdcb4df7b3db58729e6e4575` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.283824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0d4d4ccdcb4df7b3db58729e6e4575)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:32.285771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:32.349858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0d4d4ccdcb4df7b3db58729e6e4575', 'request_id': 'req-f9de6923-fb29-43dc-b032-c2d13c515c6a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:32.361813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0d4d4ccdcb4df7b3db58729e6e4575', 'request_id': 'req-f9de6923-fb29-43dc-b032-c2d13c515c6a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:32.378277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0d4d4ccdcb4df7b3db58729e6e4575', 'request_id': 'req-f9de6923-fb29-43dc-b032-c2d13c515c6a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:32.395861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] The token cache is being invalidate because project 8d0d4d4ccdcb4df7b3db58729e6e4575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:32.396552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:32.397530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f9de6923-fb29-43dc-b032-c2d13c515c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:32.399448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1043/2079] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:32 2022] DELETE /identity/v3/projects/8d0d4d4ccdcb4df7b3db58729e6e4575 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:32.404225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.405013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.405013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.433642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:32.435672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:32.507872 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1037/2080] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:32.516906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.517785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.518865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.540508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:32.542544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:32.555002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1044/2081] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:32.563243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.576406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7685d-b5da-4b69-bc17-dd186e2dfd67 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.589429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.590153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.590538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.591203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.607659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:32.607969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:32.613136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:32.651818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1038/2082] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:46:32.656078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.666568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f847d86-64ff-4e43-aae1-48b34b26b5e3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.674962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.676186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.676186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.676186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.677497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:32.679811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:32.718684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1045/2083] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:32 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:32.723265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.736650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067dc0a-e019-4b24-9ad8-559c147cdc36 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.833972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.835111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.835945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.836499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.837402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:32.840343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:32.873991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1039/2084] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:32 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:32.875497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.895971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9524d-af1d-4b92-bc59-ae44fd9b5a13 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.902611 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.903393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.903966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.904248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.904927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:32.906589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:32.913818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:32.921393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1046/2085] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:32.925200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.937427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4cbb5-1b71-4d48-b27e-5bf7bd76b558 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.946796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095daa47-9894-47e3-a298-528700c57f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.947474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095daa47-9894-47e3-a298-528700c57f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.947749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095daa47-9894-47e3-a298-528700c57f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.948020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095daa47-9894-47e3-a298-528700c57f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:32.948694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095daa47-9894-47e3-a298-528700c57f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:32.950406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095daa47-9894-47e3-a298-528700c57f53 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:32.963327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1040/2086] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:32.968322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:32.987856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd4807-b251-47b1-8c3a-5913827faa50 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:32.995359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:32.997994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:32.998302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:32.998541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] PATH_INFO: `/v3/projects/c00d2482978f4eabb4ce19af310b9d28/users/d51278528377494fbbbc6bd3308e2e2e/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:33.032618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00d2482978f4eabb4ce19af310b9d28, user_id=d51278528377494fbbbc6bd3308e2e2e, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:33.094709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:33.115206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1047/2087] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:32 2022] PUT /identity/v3/projects/c00d2482978f4eabb4ce19af310b9d28/users/d51278528377494fbbbc6bd3308e2e2e/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:33.131080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:33.131677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:33.132045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:33.195122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] MFA Rules not processed for user `d51278528377494fbbbc6bd3308e2e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:33.196611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:33.260947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:33.367520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1041/2088] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:33.371971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:33.509540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35d1d-6dd2-4d18-b245-ca0abdc605f6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:33.517024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:33.517838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:33.518217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:33.518634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:33.519726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:33.522423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:33.595256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1048/2089] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:33 2022] POST /identity/v3/projects => generated 365 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:33.599402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:33.618426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a1594-808d-4fcb-bbcc-840338ff33ad None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:33.769279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:33.770440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:33.770846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:33.771221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:33.772449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:33.777616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:33.830519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1042/2090] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:33 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:33.834457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:33.849242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13629046-b543-45f2-b360-60cbd15dcedb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:33.858233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:33.858938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:33.859345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:33.859712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:33.860384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:33.862527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:33.883736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1049/2091] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:33.888158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:33.906285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22f7ef-5637-422b-b87f-5f5b5826c2e5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:33.915929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:33.916303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:33.916571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:33.917976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] PATH_INFO: `/v3/projects/4937dd9cd3734ee29cba337859968dc3/users/40f168b6560f408f9c17bd658333fceb/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:33.941930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4937dd9cd3734ee29cba337859968dc3, user_id=40f168b6560f408f9c17bd658333fceb, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:33.945071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:33.983915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1043/2092] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:33 2022] PUT /identity/v3/projects/4937dd9cd3734ee29cba337859968dc3/users/40f168b6560f408f9c17bd658333fceb/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:33.987766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:34.000156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facda13-82a4-4809-9008-26e992e7abd9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:34.098406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e3719-b716-4b20-87b1-150b7b84b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:34.098753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e3719-b716-4b20-87b1-150b7b84b4fb admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:34.099026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e3719-b716-4b20-87b1-150b7b84b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:34.099304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e3719-b716-4b20-87b1-150b7b84b4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:34.100599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e3719-b716-4b20-87b1-150b7b84b4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:34.102269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e3719-b716-4b20-87b1-150b7b84b4fb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:34.123497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1050/2093] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:34.126733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:34.141257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8b2f-a34a-47d4-99a3-bbacad5e6c57 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:34.148528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:34.149486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:34.149918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:34.150338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] PATH_INFO: `/v3/projects/4937dd9cd3734ee29cba337859968dc3/users/40f168b6560f408f9c17bd658333fceb/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:34.153384 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4937dd9cd3734ee29cba337859968dc3, user_id=40f168b6560f408f9c17bd658333fceb, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:34.157007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:34.173822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1044/2094] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:34 2022] PUT /identity/v3/projects/4937dd9cd3734ee29cba337859968dc3/users/40f168b6560f408f9c17bd658333fceb/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:34.184948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:34.185651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:34.186321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:34.293210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] MFA Rules not processed for user `40f168b6560f408f9c17bd658333fceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:34.295090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:34.409288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:34.495516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1051/2095] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:34.508211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:34.510515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:34.638097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da438487-6fe1-4da6-8206-c5f0803272ef admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:34.646545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:34.647449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:34.647916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:34.649205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:34.666386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:34.668102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:34.707635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1045/2096] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:34.870586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:34.872338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:34.881249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:34.882839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:34.913702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46fc02-ba0b-4b68-8bf6-13b63c1e4ef3 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:34.925990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f0ab2-f485-408c-b4bc-fccc4b246b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:34.928035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0ab2-f485-408c-b4bc-fccc4b246b8f service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:34.928745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0ab2-f485-408c-b4bc-fccc4b246b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:34.929219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0ab2-f485-408c-b4bc-fccc4b246b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:34.968870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:35.082436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f0ab2-f485-408c-b4bc-fccc4b246b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:35.082986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f0ab2-f485-408c-b4bc-fccc4b246b8f service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:35.088823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06005c9-c8bf-497e-9e3a-aeffab0fd456 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:35.100760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1046/2097] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:35.103376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:35.104136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:35.104418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:35.104690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] PATH_INFO: `/v3/users/07e2276bb3804576b97dbb9a16c44d08` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:35.105925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e2276bb3804576b97dbb9a16c44d08)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:35.107931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:35.187857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e2276bb3804576b97dbb9a16c44d08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:35.215012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e2276bb3804576b97dbb9a16c44d08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:35.233963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e2276bb3804576b97dbb9a16c44d08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:35.234688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e2276bb3804576b97dbb9a16c44d08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:35.256727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e2276bb3804576b97dbb9a16c44d08'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:35.266153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1052/2098] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:34 2022] DELETE /identity/v3/users/07e2276bb3804576b97dbb9a16c44d08 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:35.270986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:35.271277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:35.271582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:35.320203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:35.321715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:35.484216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1047/2099] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:35.505001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:35.507231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:35.607918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9145a-f741-46a8-b20a-35bb4df16734 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:35.614761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26cf9d-8b7f-42f3-861f-bc1e1a36ef20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:35.615711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26cf9d-8b7f-42f3-861f-bc1e1a36ef20 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:35.616249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26cf9d-8b7f-42f3-861f-bc1e1a36ef20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:35.616734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26cf9d-8b7f-42f3-861f-bc1e1a36ef20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:35.641860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:35.651466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26cf9d-8b7f-42f3-861f-bc1e1a36ef20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:35.665309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26cf9d-8b7f-42f3-861f-bc1e1a36ef20 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:35.709942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1053/2100] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:36.636023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:36.639809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:36.654993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9aead-9bb3-4d94-aae1-dd367c5c8276 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:36.660405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:36.661173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:36.661453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:36.661721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] PATH_INFO: `/v3/projects/ef75ee4c38f14290a6c11fcbc7458318` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:36.662957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef75ee4c38f14290a6c11fcbc7458318)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:36.665142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:36.734707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef75ee4c38f14290a6c11fcbc7458318', 'request_id': 'req-e2d10be6-973a-4231-90a0-affc2e243b55'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:36.749768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef75ee4c38f14290a6c11fcbc7458318', 'request_id': 'req-e2d10be6-973a-4231-90a0-affc2e243b55'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:36.761371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef75ee4c38f14290a6c11fcbc7458318', 'request_id': 'req-e2d10be6-973a-4231-90a0-affc2e243b55'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:36.781222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] The token cache is being invalidate because project ef75ee4c38f14290a6c11fcbc7458318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:36.781753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:36.782674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2d10be6-973a-4231-90a0-affc2e243b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:36.784795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1048/2101] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:36 2022] DELETE /identity/v3/projects/ef75ee4c38f14290a6c11fcbc7458318 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:36.789789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:36.790314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:36.790314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:36.819682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:36.821220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:36.931369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1054/2102] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:36.942690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:36.943215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:36.944693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:36.969874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:36.970804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:36.981511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1049/2103] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:36.987164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.011370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c2b6b-a022-40c4-9b35-c5b3c74a16ec None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.023521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.029631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.029631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.029631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.029631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:37.029631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.029631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.044367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1055/2104] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:46:37.049410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.061574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c0b6d-17c3-45e9-9748-2a9440bc0ff3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.069733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.069733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.069733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.070245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.076504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.076504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.114763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1050/2105] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:37 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:37.119004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.135249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081580-b5f1-41c2-a64b-7ac31f99e546 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.217682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.218322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.223128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.223505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.224494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.226895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.260662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1056/2106] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:37 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:46:37.264893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.279219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550439-86fe-4808-8225-5d375da85e06 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.292636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.293877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.294316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.295077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.295757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.297370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.310812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1051/2107] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:37.314769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.329192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1546-4f06-46d0-a956-3b78b7f99a00 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.342537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb7ad7-4a6c-4add-926b-9ad1de454661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.344691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb7ad7-4a6c-4add-926b-9ad1de454661 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.344976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb7ad7-4a6c-4add-926b-9ad1de454661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.345247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb7ad7-4a6c-4add-926b-9ad1de454661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.345910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb7ad7-4a6c-4add-926b-9ad1de454661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.348399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb7ad7-4a6c-4add-926b-9ad1de454661 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.369614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1057/2108] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:37.379091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.390270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:37.398332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b63608-ff20-4fcc-b92e-82c7cc8cb4bb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.410751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f86ae5-41c7-492e-bf81-c5792efe8751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.411902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f86ae5-41c7-492e-bf81-c5792efe8751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.412358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f86ae5-41c7-492e-bf81-c5792efe8751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.412649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f86ae5-41c7-492e-bf81-c5792efe8751 admin admin] PATH_INFO: `/v3/projects/5bb4205d1965457b8a934f750d065235/users/fad3f7aff30e40d3a14071f4b002c8ce/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.436470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f86ae5-41c7-492e-bf81-c5792efe8751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb4205d1965457b8a934f750d065235, user_id=fad3f7aff30e40d3a14071f4b002c8ce, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.439247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f86ae5-41c7-492e-bf81-c5792efe8751 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.443731 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.444138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.444492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.458776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1052/2109] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:37 2022] PUT /identity/v3/projects/5bb4205d1965457b8a934f750d065235/users/fad3f7aff30e40d3a14071f4b002c8ce/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:37.469871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.470159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.470510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.487726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] MFA Rules not processed for user `b590de08f34d4265a78860006a804f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:37.489535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:37.507245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] MFA Rules not processed for user `fad3f7aff30e40d3a14071f4b002c8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:37.508454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:37.564783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:37.601984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1053/2110] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:37.618126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.620248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:37.625816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1058/2111] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:46:37.639471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.642496 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:37.706487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad08c07-ff47-438f-a450-ee2e67012ed5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.710364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.711074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.711469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.711822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.719284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66c264-f4af-4dd5-9e58-09b453ac8ec3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.722320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.723309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.723835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.723964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.724122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.724381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.726356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:37.734727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1054/2112] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:37.775832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:37.797316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.798785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7288e4-0eed-42f9-afa3-1a7fe59c2fcc service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.816893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1059/2113] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:37.858889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d03a0-7e8a-4683-8624-fc852fcbafde service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:37.863779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:37.864461 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:37.864733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:37.865317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] PATH_INFO: `/v3/users/fad3f7aff30e40d3a14071f4b002c8ce` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:37.866158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad3f7aff30e40d3a14071f4b002c8ce)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:37.868177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:37.882827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:37.954839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad3f7aff30e40d3a14071f4b002c8ce'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:37.965620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad3f7aff30e40d3a14071f4b002c8ce'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:37.966527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad3f7aff30e40d3a14071f4b002c8ce'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:37.967191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad3f7aff30e40d3a14071f4b002c8ce'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:37.975527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-54783ffb-e3cf-406a-8f76-d68803388428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad3f7aff30e40d3a14071f4b002c8ce'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:37.999332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1055/2114] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:37 2022] DELETE /identity/v3/users/fad3f7aff30e40d3a14071f4b002c8ce => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:38.005040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:38.005848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:38.005955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:38.015025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:38.015323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:38.015665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:38.027752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:38.029692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:38.046058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:38.047398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:38.147110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1060/2115] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:38.159845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1056/2116] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:46:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:38.162278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:38.164275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:38.209739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:38.211357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:38.219044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:38.238519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e13a8-62c4-4300-9cb9-34060fe222c1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:38.245544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:38.247316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:38.247316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:38.247734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:38.263291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:38.264848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:38.275820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1061/2117] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:38.292665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8cadf-aec6-4250-9941-190e26eb7790 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:38.299063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:38.300163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:38.300299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:38.300582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:38.311025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:38.312810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:38.321890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1057/2118] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:46:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 19:46:38.710803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:38.712510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:38.749179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e58e3-3174-46a1-976d-1ce659a9b990 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:38.779006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:38.780101 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:38.780609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:38.781072 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] PATH_INFO: `/v3/projects/5bb4205d1965457b8a934f750d065235` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:38.782373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb4205d1965457b8a934f750d065235)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:38.784707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:38.861043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb4205d1965457b8a934f750d065235', 'request_id': 'req-24a0f860-4ffe-42d7-8029-653a00682ffc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:38.872491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb4205d1965457b8a934f750d065235', 'request_id': 'req-24a0f860-4ffe-42d7-8029-653a00682ffc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:38.883067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb4205d1965457b8a934f750d065235', 'request_id': 'req-24a0f860-4ffe-42d7-8029-653a00682ffc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:38.905329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] The token cache is being invalidate because project 5bb4205d1965457b8a934f750d065235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:38.905626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:38.906599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-24a0f860-4ffe-42d7-8029-653a00682ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:38.928511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1062/2119] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:38 2022] DELETE /identity/v3/projects/5bb4205d1965457b8a934f750d065235 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:43.529396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:43.531165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:43.554297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44ed6-eeac-4dce-89a1-8e3eb5680714 service swift] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:43.591862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:43.649539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:43.650189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:43.650472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:43.650746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] PATH_INFO: `/v3/users/d51278528377494fbbbc6bd3308e2e2e` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:43.653398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d51278528377494fbbbc6bd3308e2e2e)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:43.656906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:43.750410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd51278528377494fbbbc6bd3308e2e2e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:43.768821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd51278528377494fbbbc6bd3308e2e2e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:43.769420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd51278528377494fbbbc6bd3308e2e2e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:43.770228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd51278528377494fbbbc6bd3308e2e2e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:43.780584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd51278528377494fbbbc6bd3308e2e2e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:43.810446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1058/2120] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:43 2022] DELETE /identity/v3/users/d51278528377494fbbbc6bd3308e2e2e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:43.812849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:43.813388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:43.814503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:43.893260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:43.894535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:44.008023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1063/2121] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:44.026840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:44.028613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:44.144379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f54b4-7adb-41b8-8980-066f9016523e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:44.149328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:44.150417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:44.151073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:44.151591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:44.170590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:44.172332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:44.200185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1059/2122] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:44.586551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:44.588102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:44.613682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eac3f7-effa-416d-85ce-8ceab0b25f74 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:44.631994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:44.632849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:44.633319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:44.633761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] PATH_INFO: `/v3/projects/c00d2482978f4eabb4ce19af310b9d28` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:44.642330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00d2482978f4eabb4ce19af310b9d28)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:44.644819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:44.733855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00d2482978f4eabb4ce19af310b9d28', 'request_id': 'req-d58ea351-a230-4c12-97f7-92ff97d39054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.742205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00d2482978f4eabb4ce19af310b9d28', 'request_id': 'req-d58ea351-a230-4c12-97f7-92ff97d39054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.750566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:44.753106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00d2482978f4eabb4ce19af310b9d28', 'request_id': 'req-d58ea351-a230-4c12-97f7-92ff97d39054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.768561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] The token cache is being invalidate because project c00d2482978f4eabb4ce19af310b9d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:44.769100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.770044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.771740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1064/2123] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:44 2022] DELETE /identity/v3/projects/c00d2482978f4eabb4ce19af310b9d28 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:44.775880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:44.789067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c61ab-5949-4939-ad9c-69f8d1f68a67 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:44.851950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:44.868536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:44.869321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:44.869692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:44.870055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] PATH_INFO: `/v3/users/40f168b6560f408f9c17bd658333fceb` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:44.873642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40f168b6560f408f9c17bd658333fceb)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:44.875769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:44.963292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40f168b6560f408f9c17bd658333fceb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.979109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40f168b6560f408f9c17bd658333fceb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.980154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40f168b6560f408f9c17bd658333fceb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.982014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40f168b6560f408f9c17bd658333fceb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:44.993708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-91e200b5-eb1b-49ec-b5bf-4b1a65d14525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40f168b6560f408f9c17bd658333fceb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:45.022087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1060/2124] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:44 2022] DELETE /identity/v3/users/40f168b6560f408f9c17bd658333fceb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:46:45.401217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:45.403823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:45.423876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea351-a230-4c12-97f7-92ff97d39054 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:45.500536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:45.501444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:45.501717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:45.502019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] PATH_INFO: `/v3/projects/4937dd9cd3734ee29cba337859968dc3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:45.503169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4937dd9cd3734ee29cba337859968dc3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:45.505155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:45.564799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4937dd9cd3734ee29cba337859968dc3', 'request_id': 'req-34231ee7-f026-44d2-9d36-8b3705c2bd3d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:45.573252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4937dd9cd3734ee29cba337859968dc3', 'request_id': 'req-34231ee7-f026-44d2-9d36-8b3705c2bd3d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:45.585064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4937dd9cd3734ee29cba337859968dc3', 'request_id': 'req-34231ee7-f026-44d2-9d36-8b3705c2bd3d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:45.615801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] The token cache is being invalidate because project 4937dd9cd3734ee29cba337859968dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:45.616738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:45.621307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-34231ee7-f026-44d2-9d36-8b3705c2bd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:45.628576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1065/2125] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:45 2022] DELETE /identity/v3/projects/4937dd9cd3734ee29cba337859968dc3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:45.641271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:45.641271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:45.641271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:45.677800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:45.679050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:45.878513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1061/2126] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:45.917859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ee133-8270-461b-afbc-004ba128274d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:45.918431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ee133-8270-461b-afbc-004ba128274d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:45.918881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ee133-8270-461b-afbc-004ba128274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:45.945823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-119ee133-8270-461b-afbc-004ba128274d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:45.947189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-119ee133-8270-461b-afbc-004ba128274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:45.969130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1066/2127] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:45.969259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:45.978592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865710-37e4-4a09-a226-047ac4ed4008 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:45.984065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:45.984803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:45.985081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:45.985364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:45.986152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:45.986456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:45.988118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.005700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1062/2128] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:46:46.008888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ee133-8270-461b-afbc-004ba128274d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:46.019898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ee133-8270-461b-afbc-004ba128274d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:46.024824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:46.025531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.025813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.026222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.026938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:46.030567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.071904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1067/2129] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:46 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:46.074364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:46.088346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098097f-1efe-4e3f-b925-17573d392c9a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:46.192063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:46.220079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:46.220939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.221281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.221919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.222659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:46.224868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.258312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1063/2130] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:46 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:46.262744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:46.273727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c423c1-1b6d-4621-a184-2645a2f774f2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:46.279488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:46.281778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.282128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.282468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.283201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:46.284957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.303199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1068/2131] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:46.307316 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:46.322741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcce7da-6daa-4d19-8d97-dc809211ce6d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:46.329114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd05c3b-b3eb-4e0d-9f13-7c246700dacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:46.330041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd05c3b-b3eb-4e0d-9f13-7c246700dacd admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.330494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd05c3b-b3eb-4e0d-9f13-7c246700dacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.330852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd05c3b-b3eb-4e0d-9f13-7c246700dacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.331632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd05c3b-b3eb-4e0d-9f13-7c246700dacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:46.333453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd05c3b-b3eb-4e0d-9f13-7c246700dacd admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.357340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1064/2132] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:46:46.358776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:46.372242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc39f29-68fe-4e6a-bce3-066b161218f8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:46.393663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:46.394585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.395213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.395615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] PATH_INFO: `/v3/projects/eba56aaf79c343abaec56d0da87a418b/users/cdc1066d58fa43a59340d083e51489ec/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.431966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba56aaf79c343abaec56d0da87a418b, user_id=cdc1066d58fa43a59340d083e51489ec, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:46.435907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.475555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1069/2133] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:46 2022] PUT /identity/v3/projects/eba56aaf79c343abaec56d0da87a418b/users/cdc1066d58fa43a59340d083e51489ec/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:46.490341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.490632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.490903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.540373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] MFA Rules not processed for user `cdc1066d58fa43a59340d083e51489ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:46.541575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:46.644088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1065/2134] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:46.659543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:46.661817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:46.684493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:46.761263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b299c-65f2-4f9c-8af1-abf3ceb710ba admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:46.768480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd7cb4-6678-41e6-9754-69fa712c9df4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:46.769215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd7cb4-6678-41e6-9754-69fa712c9df4 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:46.770289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd7cb4-6678-41e6-9754-69fa712c9df4 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:46.770567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd7cb4-6678-41e6-9754-69fa712c9df4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:46.782903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd7cb4-6678-41e6-9754-69fa712c9df4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:46.784406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd7cb4-6678-41e6-9754-69fa712c9df4 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:46.796658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1070/2135] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:49.339268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:49.346073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:49.442633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788ddd4-b69e-4158-846f-537e32958f38 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:49.449915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:49.450706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:49.451068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:49.451399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] PATH_INFO: `/v3/users/57b529553a4c44248d7323a1113f1b4d` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:49.452679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b529553a4c44248d7323a1113f1b4d)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:49.454694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:49.530075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b529553a4c44248d7323a1113f1b4d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:49.539803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b529553a4c44248d7323a1113f1b4d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:49.540656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b529553a4c44248d7323a1113f1b4d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:49.541282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b529553a4c44248d7323a1113f1b4d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:49.552463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b529553a4c44248d7323a1113f1b4d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:49.553792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:49.590446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1066/2136] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:46:49 2022] DELETE /identity/v3/users/57b529553a4c44248d7323a1113f1b4d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:49.594984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96048f-dc0b-4724-8923-57e84592d2cc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:49.595633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96048f-dc0b-4724-8923-57e84592d2cc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:49.595733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96048f-dc0b-4724-8923-57e84592d2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:49.624233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5a96048f-dc0b-4724-8923-57e84592d2cc None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:49.625695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5a96048f-dc0b-4724-8923-57e84592d2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:49.749042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1071/2137] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:49.759588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:49.766317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:49.845211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfed8a-41e2-4cf3-9d3b-8bbb8839670b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:49.849003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:49.849686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:49.849976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:49.850244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:49.864318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:49.865856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:49.885679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1067/2138] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:50.135700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a4b8-06ba-451e-8d82-8e3a47f602a2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.135700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a4b8-06ba-451e-8d82-8e3a47f602a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.135700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a4b8-06ba-451e-8d82-8e3a47f602a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.158839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-30f8a4b8-06ba-451e-8d82-8e3a47f602a2 None None] MFA Rules not processed for user `47773b38da48458fa356f34d259d3478`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:50.160422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-30f8a4b8-06ba-451e-8d82-8e3a47f602a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:50.234753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1072/2139] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:50.311655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.313260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:50.330664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a1bc6-5cb5-496b-869a-33ce0273ace1 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.336420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.337240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.337538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.337808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] PATH_INFO: `/v3/projects/92d59ea2b397495eb68a79cf433e21c7` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.339017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d59ea2b397495eb68a79cf433e21c7)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:50.341345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:50.403929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d59ea2b397495eb68a79cf433e21c7', 'request_id': 'req-4b7f3333-7b66-4b62-9608-2e808982842c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:50.415167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d59ea2b397495eb68a79cf433e21c7', 'request_id': 'req-4b7f3333-7b66-4b62-9608-2e808982842c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:50.422183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d59ea2b397495eb68a79cf433e21c7', 'request_id': 'req-4b7f3333-7b66-4b62-9608-2e808982842c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:50.437995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] The token cache is being invalidate because project 92d59ea2b397495eb68a79cf433e21c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:46:50.438326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:50.438950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4b7f3333-7b66-4b62-9608-2e808982842c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:46:50.444901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1068/2140] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:46:50 2022] DELETE /identity/v3/projects/92d59ea2b397495eb68a79cf433e21c7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:50.446528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.446861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.447163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.474498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:50.475748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:50.557564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1073/2141] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:50.566488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.566883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.567227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.591553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:50.593533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:50.601961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1069/2142] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:50.606815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.613057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:50.617879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceed77-af4a-491a-9b73-15ca039c1818 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.624171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.624793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.625068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.625379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.626016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:46:50.626367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:50.628019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:50.647486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1074/2143] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:46:50.651034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.662439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d0f1d-e351-4924-825e-64278eb194dc None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.668694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.669351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.669894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.670138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.671291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:50.672842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:50.716572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1070/2144] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:46:50 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:50.721297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.732892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9423f-958e-4190-89cc-ad1da37c0136 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.825919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.826629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.826963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.827340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.828081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:50.830204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:50.859637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1075/2145] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:46:50 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:46:50.864081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.878908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1cf18-536f-4cfd-bdcf-c2f076f0bff5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.884069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.885091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.885091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.885433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.886017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:50.888212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:50.901417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1071/2146] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:50.906189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.921400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8ff00-42f5-4f27-9996-6076d52448ba admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.931750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bfad0-8e26-417e-b660-281584ce59b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.932415 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bfad0-8e26-417e-b660-281584ce59b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.932898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bfad0-8e26-417e-b660-281584ce59b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.933016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bfad0-8e26-417e-b660-281584ce59b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:50.933666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bfad0-8e26-417e-b660-281584ce59b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:50.935464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bfad0-8e26-417e-b660-281584ce59b2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:50.953200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1076/2147] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:46:50.956821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:50.975402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3dc42-ce97-4bfe-b503-49eb3bc156b7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:50.984386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:50.985310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:50.985590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:50.985870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] PATH_INFO: `/v3/projects/f5c3250304054594abbaa610e49da216/users/e6cc144c0d804a84a8d35eaa57fab6bb/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:51.003144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c3250304054594abbaa610e49da216, user_id=e6cc144c0d804a84a8d35eaa57fab6bb, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:51.006242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:51.025982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1072/2148] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:46:50 2022] PUT /identity/v3/projects/f5c3250304054594abbaa610e49da216/users/e6cc144c0d804a84a8d35eaa57fab6bb/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:46:51.031834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afde547-422b-46d3-b42f-14d83d3d7198 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:51.032120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afde547-422b-46d3-b42f-14d83d3d7198 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:51.032391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afde547-422b-46d3-b42f-14d83d3d7198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:51.066154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5afde547-422b-46d3-b42f-14d83d3d7198 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:51.069515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5afde547-422b-46d3-b42f-14d83d3d7198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:51.155655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1077/2149] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:51.166573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:51.170223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:51.175132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:51.254314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdfd8-937f-4c5a-bfe2-46dcdab583c1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:51.257004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488087c4-4c71-43c7-b434-57f6a694e6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:51.257636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488087c4-4c71-43c7-b434-57f6a694e6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:51.259275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488087c4-4c71-43c7-b434-57f6a694e6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:51.259584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488087c4-4c71-43c7-b434-57f6a694e6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:51.269796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488087c4-4c71-43c7-b434-57f6a694e6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:51.271288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488087c4-4c71-43c7-b434-57f6a694e6a5 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:51.283891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1073/2150] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:46:55.081116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:55.086661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:55.086868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:55.122862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] MFA Rules not processed for user `be2ad8e7eefe447ca1374d49696b1318`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:55.124429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:55.235173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1078/2151] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:46:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:55.396184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fca5c0-231f-49b2-a591-ad971d796d0e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:55.397258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fca5c0-231f-49b2-a591-ad971d796d0e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:55.397739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fca5c0-231f-49b2-a591-ad971d796d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:55.439352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e2fca5c0-231f-49b2-a591-ad971d796d0e None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:46:55.441039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e2fca5c0-231f-49b2-a591-ad971d796d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:55.577440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:46:55.578096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1074/2152] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:46:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:46:55.580302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:46:55.619162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:46:55.757862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a193b-1266-44d6-9017-8939331beb96 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:46:55.765182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:46:55.765816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:46:55.766092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:46:55.766358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:46:55.792181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:46:55.794171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:46:55.813613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1079/2153] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:46:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 19:47:00.816444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:00.817430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:00.818088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:00.859996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] MFA Rules not processed for user `e6cc144c0d804a84a8d35eaa57fab6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:00.862243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:01.021501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1075/2154] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:01.026643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:01.048809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cca7b-dda9-437f-a3ea-ea3e94512d39 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:01.066113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:01.070669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:01.071316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:01.074096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:01.076212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:01.082703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:01.178908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1080/2155] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:47:01 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:47:01.182865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:01.209822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed804a-6a0e-4f92-814a-41dc4c5dd4f8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:01.354225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:01.412120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:01.412749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:01.413058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:01.413393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:01.414050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:01.425291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:01.487102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1076/2156] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:47:01 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:47:01.491123 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:01.541250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c54946-9221-430f-be74-588f1dda011e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:01.571481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:01.572524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:01.572999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:01.573688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:01.574595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:01.580187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:01.648737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1081/2157] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:47:01.654133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:01.689879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274e30c-5648-42cb-af55-3d4ad070f4a9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:01.723859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:01.724653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:01.725008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:01.725268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] PATH_INFO: `/v3/projects/916b19fbba7d4e16ac17b6057c07180e/users/977642e6e8404054943b619524e5078b/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:01.786870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=916b19fbba7d4e16ac17b6057c07180e, user_id=977642e6e8404054943b619524e5078b, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:01.789760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:01.849426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1077/2158] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:47:01 2022] PUT /identity/v3/projects/916b19fbba7d4e16ac17b6057c07180e/users/977642e6e8404054943b619524e5078b/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:01.850940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:01.869794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a7db-aea5-46c5-847c-f7bf02b4ba26 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:02.043659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:02.044843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:02.044843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:02.045092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:02.046008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:02.052759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:02.095388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1082/2159] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:47:02.099340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:02.126123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adb96e-29c5-4825-8472-4a95a404e8bd admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:02.144633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e0628-6920-4cff-b9c6-a07e0deb217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:02.145591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e0628-6920-4cff-b9c6-a07e0deb217b admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:02.146048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e0628-6920-4cff-b9c6-a07e0deb217b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:02.146364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e0628-6920-4cff-b9c6-a07e0deb217b admin admin] PATH_INFO: `/v3/projects/916b19fbba7d4e16ac17b6057c07180e/users/977642e6e8404054943b619524e5078b/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:02.150813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e0628-6920-4cff-b9c6-a07e0deb217b admin admin] RBAC: Authorizing `identity:create_grant(project_id=916b19fbba7d4e16ac17b6057c07180e, user_id=977642e6e8404054943b619524e5078b, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:02.154614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e0628-6920-4cff-b9c6-a07e0deb217b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:02.193630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1078/2160] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:47:02 2022] PUT /identity/v3/projects/916b19fbba7d4e16ac17b6057c07180e/users/977642e6e8404054943b619524e5078b/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:07.414829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:07.420015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:07.458724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450dcb4-62ea-479b-9b8e-c82cba2d792f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:07.646828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:07.649410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:07.650649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:07.653624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] PATH_INFO: `/v3/users/cdc1066d58fa43a59340d083e51489ec` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:07.657732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc1066d58fa43a59340d083e51489ec)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:07.661377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:07.732322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:07.845898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc1066d58fa43a59340d083e51489ec'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:07.894927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc1066d58fa43a59340d083e51489ec'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:07.926833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc1066d58fa43a59340d083e51489ec'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:07.928430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc1066d58fa43a59340d083e51489ec'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:07.950693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc1066d58fa43a59340d083e51489ec'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:07.965401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1083/2161] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:47:07 2022] DELETE /identity/v3/users/cdc1066d58fa43a59340d083e51489ec => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:07.971734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc79e-8565-49a6-b806-8469b197427c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:07.972735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc79e-8565-49a6-b806-8469b197427c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:07.973243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc79e-8565-49a6-b806-8469b197427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:08.015770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-201dc79e-8565-49a6-b806-8469b197427c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:08.018943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-201dc79e-8565-49a6-b806-8469b197427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:08.173547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1079/2162] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:08.186022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:08.191471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:08.359695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3fd2-3feb-4e56-a3b5-8a6dc5434d6d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:08.369471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5d04e-f974-4c84-8d27-8d153ed1b441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:08.370104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5d04e-f974-4c84-8d27-8d153ed1b441 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:08.374394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5d04e-f974-4c84-8d27-8d153ed1b441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:08.374954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5d04e-f974-4c84-8d27-8d153ed1b441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:08.404547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5d04e-f974-4c84-8d27-8d153ed1b441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:08.407852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5d04e-f974-4c84-8d27-8d153ed1b441 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:08.454396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1084/2163] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:09.734068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dc79e-8565-49a6-b806-8469b197427c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:09.737060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-201dc79e-8565-49a6-b806-8469b197427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:09.762939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dc79e-8565-49a6-b806-8469b197427c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:09.769233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:09.770199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:09.770638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:09.771061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] PATH_INFO: `/v3/projects/eba56aaf79c343abaec56d0da87a418b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:09.773136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba56aaf79c343abaec56d0da87a418b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:09.779278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:10.025123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba56aaf79c343abaec56d0da87a418b', 'request_id': 'req-7e2ec20a-43e8-4662-a965-c84874e1e32a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:10.043878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba56aaf79c343abaec56d0da87a418b', 'request_id': 'req-7e2ec20a-43e8-4662-a965-c84874e1e32a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:10.097496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba56aaf79c343abaec56d0da87a418b', 'request_id': 'req-7e2ec20a-43e8-4662-a965-c84874e1e32a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:10.125314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] The token cache is being invalidate because project eba56aaf79c343abaec56d0da87a418b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:47:10.125887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:10.128646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7e2ec20a-43e8-4662-a965-c84874e1e32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:10.139981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1080/2164] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:47:09 2022] DELETE /identity/v3/projects/eba56aaf79c343abaec56d0da87a418b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:10.141681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.142203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.142203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.180473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:10.187297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:10.304566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1085/2165] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:10.305833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.306115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.307635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.361038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:10.364225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:10.391497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1081/2166] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:47:10.395782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:10.431759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169908f2-a4d5-43db-8191-6d1147a53af6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:10.446565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:10.447356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.448331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.449655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.450866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:47:10.451949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:10.454366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:10.498805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:10.507182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1086/2167] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:47:10.518882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268449-2cdb-471e-b62a-e7956deedfc0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:10.530082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:10.531327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.532201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.551471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.553998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:10.566791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:10.584881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:10.668420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1082/2168] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:47:10 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:47:10.670314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:10.688254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4fea7-03a0-468a-bb6a-a1df9a497ad1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:10.722235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:10.796463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:10.797499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.797690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.798115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.798865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:10.801184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:10.837012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1087/2169] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:47:10 2022] POST /identity/v3/users => generated 281 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:47:10.838222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:10.856368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070848e-adb6-4fd3-8bb8-fbd60e619323 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:10.862205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:10.862829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.863493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.864005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.864678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:10.866540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:10.878625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1083/2170] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:47:10.883005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:10.896159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee16a15-26fb-4348-97e2-5db762fbc026 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:10.906798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23324-887c-46a0-8b5b-038d5e06ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:10.907607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23324-887c-46a0-8b5b-038d5e06ca24 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.907945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23324-887c-46a0-8b5b-038d5e06ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.908224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23324-887c-46a0-8b5b-038d5e06ca24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.908978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d23324-887c-46a0-8b5b-038d5e06ca24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:10.911024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d23324-887c-46a0-8b5b-038d5e06ca24 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:10.933223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1088/2171] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:47:10.933828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:10.950550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ecec4-b4c8-45f0-8f5c-ebdc3be0b814 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:10.959444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:10.961285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:10.961728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:10.962868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] PATH_INFO: `/v3/projects/99601610c7cc40c1a883a4ce0a466883/users/b6ef7226fc804e059ad07bb969eb863b/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:10.992737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99601610c7cc40c1a883a4ce0a466883, user_id=b6ef7226fc804e059ad07bb969eb863b, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:10.997032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:11.076236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1084/2172] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:47:10 2022] PUT /identity/v3/projects/99601610c7cc40c1a883a4ce0a466883/users/b6ef7226fc804e059ad07bb969eb863b/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:47:11.108222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e29e-1d09-4123-a2cc-f6d107b5394c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:11.108631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e29e-1d09-4123-a2cc-f6d107b5394c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:11.108915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e29e-1d09-4123-a2cc-f6d107b5394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:11.177393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-64f9e29e-1d09-4123-a2cc-f6d107b5394c None None] MFA Rules not processed for user `b6ef7226fc804e059ad07bb969eb863b`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:11.180646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-64f9e29e-1d09-4123-a2cc-f6d107b5394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:11.337747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1089/2173] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:11.383143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:11.388560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:11.559042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b9d4-421f-4209-95f9-f3051f548169 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:11.575399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:11.576304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:11.576771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:11.577704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:11.582374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:11.582915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:11.583351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:11.618635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:11.622648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:11.645992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1085/2174] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:11.647595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] MFA Rules not processed for user `977642e6e8404054943b619524e5078b`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:11.648898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:11.662924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:11.858294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185134-5ddb-41a0-a1b5-68828908b680 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:11.860572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1090/2175] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:47:11.864676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:11.865882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:11.866274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:11.866675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] PATH_INFO: `/v3/users/b6ef7226fc804e059ad07bb969eb863b` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:11.868391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ef7226fc804e059ad07bb969eb863b)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:11.871248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:11.874994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:11.876282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:11.900501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:11.995165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ef7226fc804e059ad07bb969eb863b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:12.013387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ef7226fc804e059ad07bb969eb863b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:12.014428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ef7226fc804e059ad07bb969eb863b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:12.016748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ef7226fc804e059ad07bb969eb863b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:12.020177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:12.023391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff00f40-db10-4713-983c-699e28dc7119 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:12.028485 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-5fcfbc9e-b9bc-4b19-9916-5b7773875f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ef7226fc804e059ad07bb969eb863b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:12.030522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:12.031149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:12.031902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:12.032181 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:12.034337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:12.065099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1086/2176] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:47:11 2022] DELETE /identity/v3/users/b6ef7226fc804e059ad07bb969eb863b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:12.092836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4de97-6a2e-4d33-af35-45a16804d495 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:12.093268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4de97-6a2e-4d33-af35-45a16804d495 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:12.093687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4de97-6a2e-4d33-af35-45a16804d495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:12.119224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-94e4de97-6a2e-4d33-af35-45a16804d495 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:12.121368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-94e4de97-6a2e-4d33-af35-45a16804d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:12.133388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:12.136800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:12.152856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1091/2177] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:47:12.215506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1087/2178] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:12.227781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:12.230575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:12.325143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb93807-a090-4630-a989-0c75061ae431 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:12.399452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:12.400187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:12.400482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:12.400760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:12.413521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:12.415036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:12.433014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1092/2179] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:12.536825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd2321-5397-4d1c-b8f5-239d3600d158 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:12.537241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd2321-5397-4d1c-b8f5-239d3600d158 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:12.537725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd2321-5397-4d1c-b8f5-239d3600d158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:12.612523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1bbd2321-5397-4d1c-b8f5-239d3600d158 None None] MFA Rules not processed for user `f417a94fed274bfbb88d23677a4877a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:12.614514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1bbd2321-5397-4d1c-b8f5-239d3600d158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:12.737321 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1088/2180] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:13.749688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:13.751177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:13.768998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7a9dd-9d6b-4cb3-8d1b-8446bee16930 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:13.783324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:13.784086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:13.784394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:13.784726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] PATH_INFO: `/v3/projects/99601610c7cc40c1a883a4ce0a466883` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:13.792291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99601610c7cc40c1a883a4ce0a466883)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:13.794175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:13.796358 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:13.911292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99601610c7cc40c1a883a4ce0a466883', 'request_id': 'req-f403dd92-0d99-477a-a0d2-579a0c4caaf6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:13.929909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99601610c7cc40c1a883a4ce0a466883', 'request_id': 'req-f403dd92-0d99-477a-a0d2-579a0c4caaf6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:13.942194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99601610c7cc40c1a883a4ce0a466883', 'request_id': 'req-f403dd92-0d99-477a-a0d2-579a0c4caaf6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:13.991819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] The token cache is being invalidate because project 99601610c7cc40c1a883a4ce0a466883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:47:13.992147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:13.993650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f403dd92-0d99-477a-a0d2-579a0c4caaf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:47:13.996125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1093/2181] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:47:13 2022] DELETE /identity/v3/projects/99601610c7cc40c1a883a4ce0a466883 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:14.000812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.001341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.001341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.044027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:14.045449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:14.189803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1089/2182] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:14.191473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.191782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.192231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.222309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:14.222974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:14.235680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1094/2183] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:14.237184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:14.252320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b1025-fba3-4d60-9a43-2e4dfa72ffe3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:14.260113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:14.260958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.261361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.262403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.263219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:47:14.263922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:14.266266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:14.288279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1090/2184] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:47:14.294005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:14.307524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfeb05-edef-47f4-9bad-d2d5912dd32a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:14.315210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:14.316312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.316789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.317133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.318285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:14.321161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:14.369463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1095/2185] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:47:14 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:47:14.376418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:14.390600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59ca91-974b-40d2-8ed9-e750da766f28 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:14.404201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:14.537579 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:14.538260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.538482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.538750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.539393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:14.541377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:14.589372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1091/2186] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:47:14 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:47:14.591838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:14.617067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e8d7f-161c-469e-bf9e-6d8df610ddb3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:14.638200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:14.638835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.639853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.639853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.640046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:14.641684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:14.671587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1096/2187] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:47:14.674189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:14.688317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c903ea3-3616-4f58-a820-673d8b3c43fb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:14.702142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666603-4347-48a9-8f67-8275bbbde4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:14.703575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666603-4347-48a9-8f67-8275bbbde4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.704146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666603-4347-48a9-8f67-8275bbbde4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.706562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666603-4347-48a9-8f67-8275bbbde4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.708577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0666603-4347-48a9-8f67-8275bbbde4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:14.710576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0666603-4347-48a9-8f67-8275bbbde4f0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:14.795924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1092/2188] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:47:14.798786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:14.816818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09078-1225-40fd-aae9-350010530d7d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:14.822786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871a6d-218b-474c-a482-4d1401dfb8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:14.824018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871a6d-218b-474c-a482-4d1401dfb8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.824318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871a6d-218b-474c-a482-4d1401dfb8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.824574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871a6d-218b-474c-a482-4d1401dfb8ef admin admin] PATH_INFO: `/v3/projects/da8284d3beaa421e893bd04ba33081ea/users/7d6386b432054b0d9c9e347ec8847462/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.842882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871a6d-218b-474c-a482-4d1401dfb8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8284d3beaa421e893bd04ba33081ea, user_id=7d6386b432054b0d9c9e347ec8847462, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:14.845827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871a6d-218b-474c-a482-4d1401dfb8ef admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:14.864338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1097/2189] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:47:14 2022] PUT /identity/v3/projects/da8284d3beaa421e893bd04ba33081ea/users/7d6386b432054b0d9c9e347ec8847462/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:47:14.869717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:14.870516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:14.870516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:14.899320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:14.901200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:15.001518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e21f3-689e-480c-9c3c-840a84391528 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:15.001753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e21f3-689e-480c-9c3c-840a84391528 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:15.002373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e21f3-689e-480c-9c3c-840a84391528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:15.008277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1093/2190] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:15.019893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:15.020966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:15.032092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3b6e21f3-689e-480c-9c3c-840a84391528 None None] MFA Rules not processed for user `e6cc144c0d804a84a8d35eaa57fab6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:15.035529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3b6e21f3-689e-480c-9c3c-840a84391528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:15.160471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1098/2191] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:15.180441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f0bb4-77fb-484b-b86a-d117695b5580 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:15.206754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:15.207393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:15.207718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:15.207968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:15.230250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:15.231799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:15.255301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1094/2192] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:16.756781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d60f5d-8b28-49a8-95eb-119725acc820 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:16.760184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d60f5d-8b28-49a8-95eb-119725acc820 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:16.760524 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d60f5d-8b28-49a8-95eb-119725acc820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:16.871032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-44d60f5d-8b28-49a8-95eb-119725acc820 None None] MFA Rules not processed for user `7d6386b432054b0d9c9e347ec8847462`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:16.872730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-44d60f5d-8b28-49a8-95eb-119725acc820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:16.959601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:17.087551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1099/2193] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:17.107802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:17.109937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:17.200812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:17.278485 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b002a7-7369-40fa-b863-7da20b2fe1ed service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:17.296107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda2eb3-5713-4aa5-8431-1a07dbc7c2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:17.296793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda2eb3-5713-4aa5-8431-1a07dbc7c2ee service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:17.297083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda2eb3-5713-4aa5-8431-1a07dbc7c2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:17.297337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda2eb3-5713-4aa5-8431-1a07dbc7c2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:17.314937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda2eb3-5713-4aa5-8431-1a07dbc7c2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:17.316898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda2eb3-5713-4aa5-8431-1a07dbc7c2ee service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:17.337553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1095/2194] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:18.967007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:18.967716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:18.967809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:18.986945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] MFA Rules not processed for user `7d6386b432054b0d9c9e347ec8847462`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:18.988205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:18.996205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1100/2195] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:27.329522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=13 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:28.981251 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=21 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:34.272855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:34.273457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:34.273457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:34.296593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:34.297858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:34.367827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1096/2196] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:47:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:34.382665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:34.384158 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:34.482574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d7487-08d1-4ab2-b25a-c7af4767e07a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:34.488325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7328fb-d397-4660-8a20-df9237ee7cfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:34.489192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7328fb-d397-4660-8a20-df9237ee7cfe service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:34.489540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7328fb-d397-4660-8a20-df9237ee7cfe service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:34.489881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7328fb-d397-4660-8a20-df9237ee7cfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:34.504861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7328fb-d397-4660-8a20-df9237ee7cfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:34.507469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7328fb-d397-4660-8a20-df9237ee7cfe service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:34.517583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1101/2197] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:47:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:47:34.663819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:34.676791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d6673-21f6-41a4-91ff-2b57fdf971d8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:34.686855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:34.687771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:34.689221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:34.689609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:34.693826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:34.820573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:34.822772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8e358-2ef1-4fb3-bcbc-8f588cd7cb72 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:34.836184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1097/2198] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:44.513669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=32 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:44.830583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=55 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:47:46.551680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ac561-c2ca-493a-81eb-a11b5593a03d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:46.552590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ac561-c2ca-493a-81eb-a11b5593a03d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:46.552875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ac561-c2ca-493a-81eb-a11b5593a03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:46.578280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-552ac561-c2ca-493a-81eb-a11b5593a03d None None] MFA Rules not processed for user `7d6386b432054b0d9c9e347ec8847462`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:46.579539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-552ac561-c2ca-493a-81eb-a11b5593a03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:46.587767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1102/2199] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:47:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:54.607046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67eb29-6e64-41ff-b2fe-2bcc8694af3e None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:54.607535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67eb29-6e64-41ff-b2fe-2bcc8694af3e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:54.607535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67eb29-6e64-41ff-b2fe-2bcc8694af3e None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:54.608908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1098/2200] 10.222.0.55 () {54 vars in 961 bytes} [Sat Sep 24 19:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:47:54.614087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88209-522f-4f1e-a4af-21276e9b24b1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:54.614545 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88209-522f-4f1e-a4af-21276e9b24b1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:54.614748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88209-522f-4f1e-a4af-21276e9b24b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:54.641904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2aa88209-522f-4f1e-a4af-21276e9b24b1 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:54.643230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2aa88209-522f-4f1e-a4af-21276e9b24b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:54.716970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1103/2201] 10.222.0.55 () {58 vars in 1058 bytes} [Sat Sep 24 19:47:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:54.724822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67eb29-6e64-41ff-b2fe-2bcc8694af3e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:54.740116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67eb29-6e64-41ff-b2fe-2bcc8694af3e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:54.746932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:54.747764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:54.748863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:54.748863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:54.750502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:54.763581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:54.765448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734d19c-74a5-4ef3-a513-80efc38f3e02 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:54.778051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1099/2202] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:57.149314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6063ef9-c0c0-4754-8bcb-7aebe4bc2924 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.152304 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6063ef9-c0c0-4754-8bcb-7aebe4bc2924 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.152691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6063ef9-c0c0-4754-8bcb-7aebe4bc2924 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.154894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1104/2203] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:47:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:47:57.169006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8732-ad6a-4b6a-9cce-7518922705c1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.169797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8732-ad6a-4b6a-9cce-7518922705c1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.169797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8732-ad6a-4b6a-9cce-7518922705c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.191960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-526d8732-ad6a-4b6a-9cce-7518922705c1 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:57.193511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-526d8732-ad6a-4b6a-9cce-7518922705c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:57.271278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1100/2204] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:47:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:57.284498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6063ef9-c0c0-4754-8bcb-7aebe4bc2924 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:57.286260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d6063ef9-c0c0-4754-8bcb-7aebe4bc2924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:57.344632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6063ef9-c0c0-4754-8bcb-7aebe4bc2924 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:57.348216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3f130-38dc-470c-adda-331395387fa2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:57.348823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3f130-38dc-470c-adda-331395387fa2 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.349100 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3f130-38dc-470c-adda-331395387fa2 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.349371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3f130-38dc-470c-adda-331395387fa2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.360233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3f130-38dc-470c-adda-331395387fa2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:57.361663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3f130-38dc-470c-adda-331395387fa2 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:57.372953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1105/2205] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:57.443640 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a84619-2ba1-4c3a-baf8-85c901179ab3 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.443850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a84619-2ba1-4c3a-baf8-85c901179ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.444116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a84619-2ba1-4c3a-baf8-85c901179ab3 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.445504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1101/2206] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:47:57.450078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d7d4-29a7-4fef-9392-ce63b0f46dad None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.450361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d7d4-29a7-4fef-9392-ce63b0f46dad None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.450629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d7d4-29a7-4fef-9392-ce63b0f46dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.470677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1fd3d7d4-29a7-4fef-9392-ce63b0f46dad None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:57.471989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1fd3d7d4-29a7-4fef-9392-ce63b0f46dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:57.479805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1106/2207] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:47:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:57.491558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a84619-2ba1-4c3a-baf8-85c901179ab3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:57.493280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-98a84619-2ba1-4c3a-baf8-85c901179ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:57.507646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a84619-2ba1-4c3a-baf8-85c901179ab3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:57.511580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab18fb-29be-4d0d-bcbf-7f9700510d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:57.512203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab18fb-29be-4d0d-bcbf-7f9700510d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.512473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab18fb-29be-4d0d-bcbf-7f9700510d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.512743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab18fb-29be-4d0d-bcbf-7f9700510d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.520275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab18fb-29be-4d0d-bcbf-7f9700510d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:57.521727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab18fb-29be-4d0d-bcbf-7f9700510d8e service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:57.530489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1102/2208] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:47:57.614450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be1fe8-817c-4426-984e-4466a780d18f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.614730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be1fe8-817c-4426-984e-4466a780d18f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.614999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be1fe8-817c-4426-984e-4466a780d18f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.616549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1107/2209] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:47:57.622693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.622973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.623242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.643118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:47:57.644477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:47:57.654122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1103/2210] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:47:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:47:57.664634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be1fe8-817c-4426-984e-4466a780d18f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:47:57.676397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be1fe8-817c-4426-984e-4466a780d18f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:47:57.680110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8150870-fb57-4892-8c56-b96a7079bfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:47:57.680704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8150870-fb57-4892-8c56-b96a7079bfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:47:57.680977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8150870-fb57-4892-8c56-b96a7079bfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:47:57.681945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8150870-fb57-4892-8c56-b96a7079bfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:47:57.692356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8150870-fb57-4892-8c56-b96a7079bfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:47:57.693886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8150870-fb57-4892-8c56-b96a7079bfe8 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:47:57.705143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1108/2211] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:48:02.240941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:02.251665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455f33-6919-48bc-9e18-f4e2b99569b2 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:02.255739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:02.256359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:02.256632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:02.256924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:02.258693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:02.326040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:02.327540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:02.337069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1104/2212] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:48:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:48:04.789271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369a29-14e9-4a45-8022-545cdfa69d4a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:04.789746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369a29-14e9-4a45-8022-545cdfa69d4a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:04.789857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369a29-14e9-4a45-8022-545cdfa69d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:04.814675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-38369a29-14e9-4a45-8022-545cdfa69d4a None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:04.816671 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-38369a29-14e9-4a45-8022-545cdfa69d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:04.824858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1109/2213] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:48:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:04.843652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:04.856362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a165e9-f1c8-472c-b6e5-81224145618f service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:04.861754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:04.862377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:04.862652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:04.862918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:04.872262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:04.874096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:04.884752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1105/2214] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:48:14.807503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=70 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:14.879623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=82 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:27.191119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f113-ac93-42ac-b7e2-b63b8ab343bf None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:27.191771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f113-ac93-42ac-b7e2-b63b8ab343bf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:27.191771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f113-ac93-42ac-b7e2-b63b8ab343bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:27.212377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8382f113-ac93-42ac-b7e2-b63b8ab343bf None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:27.213622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8382f113-ac93-42ac-b7e2-b63b8ab343bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:27.222475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1110/2215] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:48:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:27.235774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:27.247275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f418ec4-7a76-448a-83cc-fce22c75f311 service swift] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:27.252564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:27.253203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:27.253503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:27.253737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:27.262789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:27.264432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:27.272721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1106/2216] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:48:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:48:33.198638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61455-f0fa-4bfc-b9cd-075de48d1676 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:33.199262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61455-f0fa-4bfc-b9cd-075de48d1676 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:33.199361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61455-f0fa-4bfc-b9cd-075de48d1676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:33.225900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-06c61455-f0fa-4bfc-b9cd-075de48d1676 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:33.228060 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-06c61455-f0fa-4bfc-b9cd-075de48d1676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:33.237918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1111/2217] 10.222.0.55 () {58 vars in 967 bytes} [Sat Sep 24 19:48:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:33.266219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:33.286670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08930e1-9c9e-431a-87ad-9af6df757139 service swift] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:33.314892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d472fa-bc46-4259-b77e-55c9e3dc8a24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:33.315705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d472fa-bc46-4259-b77e-55c9e3dc8a24 service swift] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:33.316601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d472fa-bc46-4259-b77e-55c9e3dc8a24 service swift] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:33.317109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d472fa-bc46-4259-b77e-55c9e3dc8a24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:33.332858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d472fa-bc46-4259-b77e-55c9e3dc8a24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:33.335193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d472fa-bc46-4259-b77e-55c9e3dc8a24 service swift] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:33.349012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1107/2218] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:48:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:48:43.219105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:43.343961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=12 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:44.575819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:44.576335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:44.576335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:44.595728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] MFA Rules not processed for user `be2ad8e7eefe447ca1374d49696b1318`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:44.596929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:44.653055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1112/2219] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:48:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:45.457906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ee2a-24ca-45b6-93b7-160d89bafe27 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:45.458381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ee2a-24ca-45b6-93b7-160d89bafe27 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:45.458495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ee2a-24ca-45b6-93b7-160d89bafe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:45.479205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5d17ee2a-24ca-45b6-93b7-160d89bafe27 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:45.480826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5d17ee2a-24ca-45b6-93b7-160d89bafe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:45.488647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1108/2220] 10.222.0.55 () {58 vars in 967 bytes} [Sat Sep 24 19:48:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:45.513014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:45.525047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b8d8b-39aa-4b4c-800d-72520c2c990f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:45.533504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb97e2-203e-4100-af68-468bdbeef0de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:45.534744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb97e2-203e-4100-af68-468bdbeef0de service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:45.535200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb97e2-203e-4100-af68-468bdbeef0de service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:45.535586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb97e2-203e-4100-af68-468bdbeef0de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:45.544954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb97e2-203e-4100-af68-468bdbeef0de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:45.546555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb97e2-203e-4100-af68-468bdbeef0de service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:45.555347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1113/2221] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:48:47.587598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5fca2-fc9f-4bcf-8e6c-5c29b836cc8d None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:47.588733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5fca2-fc9f-4bcf-8e6c-5c29b836cc8d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:47.589190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5fca2-fc9f-4bcf-8e6c-5c29b836cc8d None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:47.590855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1109/2222] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:48:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:48:47.641322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:47.641866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:47.641964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:47.665815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] MFA Rules not processed for user `3a2247b0344a416788f593199de5e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:47.668917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:47.695301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5fca2-fc9f-4bcf-8e6c-5c29b836cc8d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:47.695919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1114/2223] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:48:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:47.710524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5fca2-fc9f-4bcf-8e6c-5c29b836cc8d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:47.718018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:47.719065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:47.720104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:47.720539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] PATH_INFO: `/v3/users/7d6386b432054b0d9c9e347ec8847462` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:47.721912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6386b432054b0d9c9e347ec8847462)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:47.724121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:47.756681 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:47.771661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147b142-e1e6-49f1-a189-a35ead6d12a1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:47.781737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:47.782612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:47.790906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:47.792798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:47.807172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:47.809339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:47.811356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6386b432054b0d9c9e347ec8847462'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:47.823081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1115/2224] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:48:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:48:47.837018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6386b432054b0d9c9e347ec8847462'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:47.838233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6386b432054b0d9c9e347ec8847462'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:47.838920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6386b432054b0d9c9e347ec8847462'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:47.849225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b000f88b-bca9-4a4d-adca-6224495fc2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6386b432054b0d9c9e347ec8847462'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:47.870575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1110/2225] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:48:47 2022] DELETE /identity/v3/users/7d6386b432054b0d9c9e347ec8847462 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:48:48.046479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.061583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae4270-3589-4f7a-9f61-e8669631455b service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.131144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.132021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.132306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.132739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] PATH_INFO: `/v3/projects/da8284d3beaa421e893bd04ba33081ea` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.133936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8284d3beaa421e893bd04ba33081ea)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.136173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.189793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8284d3beaa421e893bd04ba33081ea', 'request_id': 'req-71926d81-2609-47b5-82b6-e8771cf14e8f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:48.198543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8284d3beaa421e893bd04ba33081ea', 'request_id': 'req-71926d81-2609-47b5-82b6-e8771cf14e8f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:48.207676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8284d3beaa421e893bd04ba33081ea', 'request_id': 'req-71926d81-2609-47b5-82b6-e8771cf14e8f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:48.222302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] The token cache is being invalidate because project da8284d3beaa421e893bd04ba33081ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:48:48.222732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:48.223522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-71926d81-2609-47b5-82b6-e8771cf14e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:48.225305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1116/2226] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:48:48 2022] DELETE /identity/v3/projects/da8284d3beaa421e893bd04ba33081ea => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:48:48.231730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.232270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.232728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.264843 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:48.266389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:48.333482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1111/2227] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:48.343013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.343329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.343630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.352127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:48.369095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:48.370886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:48.381413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1117/2228] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:48.385517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.403028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87176b4-6df7-42ca-be6c-bb87fc204348 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.411018 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.411660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.411937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.412207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.412854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:48:48.413155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.415580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.428527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1112/2229] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:48:48.433653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.443387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aedfa5-904a-44b0-a8f6-19738bc55f6f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.448175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.448785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.449066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.449337 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.449975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.451953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.485421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1118/2230] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:48:48 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:48:48.487059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.499037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc6ee-93dd-4c44-8b3b-849adf02a1b3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.551892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:48.563769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.564589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.565076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.565617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.566307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.568426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.593328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1113/2231] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:48:48 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:48:48.594466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.605441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5f5a0-e851-40b4-b607-9a0f909750bc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.610045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.610646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.610928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.611202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.611876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.613473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.624376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1119/2232] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:48:48.628304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.640774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672ac6-4079-4ac0-a334-e798667a34f9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.646857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a756a1-5162-425a-bb1f-1da5bbef4ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.647608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a756a1-5162-425a-bb1f-1da5bbef4ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.647945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a756a1-5162-425a-bb1f-1da5bbef4ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.648833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a756a1-5162-425a-bb1f-1da5bbef4ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.649553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a756a1-5162-425a-bb1f-1da5bbef4ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.651442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a756a1-5162-425a-bb1f-1da5bbef4ac7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.667063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1114/2233] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:48:48.671235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.682605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3946d9-b37f-4b7e-a461-6ea38384dd68 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.688583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.689494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.690110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.690389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] PATH_INFO: `/v3/projects/bd2c477bc5254a79aa66ff975f8a02ad/users/9296e886f8cf42aeaf7d3bba4f0e70bb/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.713744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2c477bc5254a79aa66ff975f8a02ad, user_id=9296e886f8cf42aeaf7d3bba4f0e70bb, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.717489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.735386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1120/2234] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:48:48 2022] PUT /identity/v3/projects/bd2c477bc5254a79aa66ff975f8a02ad/users/9296e886f8cf42aeaf7d3bba4f0e70bb/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:48:48.745645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.746030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.746400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.777353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] MFA Rules not processed for user `9296e886f8cf42aeaf7d3bba4f0e70bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:48.778611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:48.877946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1115/2235] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:48:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:48.886659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:48.888758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:48.958412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85a02b-9113-429d-8be7-8b47adf1b819 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:48.964175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8ad9-4a29-4c94-ae7d-b9e43f10f81b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:48.964795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8ad9-4a29-4c94-ae7d-b9e43f10f81b service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:48.965104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8ad9-4a29-4c94-ae7d-b9e43f10f81b service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:48.965867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8ad9-4a29-4c94-ae7d-b9e43f10f81b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:48.974622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8ad9-4a29-4c94-ae7d-b9e43f10f81b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:48.977556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8ad9-4a29-4c94-ae7d-b9e43f10f81b service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:48.988494 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1121/2236] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:48:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:48:51.688239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:51.756702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06415-62e0-4c63-8f20-e01b8e7d718d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:51.761533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:51.762170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:51.762448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:51.762735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] PATH_INFO: `/v3/users/9296e886f8cf42aeaf7d3bba4f0e70bb` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:51.763874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9296e886f8cf42aeaf7d3bba4f0e70bb)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:51.765802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:51.815608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9296e886f8cf42aeaf7d3bba4f0e70bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:51.823375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9296e886f8cf42aeaf7d3bba4f0e70bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:51.824121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9296e886f8cf42aeaf7d3bba4f0e70bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:51.824744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9296e886f8cf42aeaf7d3bba4f0e70bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:51.831987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9296e886f8cf42aeaf7d3bba4f0e70bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:51.833105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:51.853062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1116/2237] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:48:51 2022] DELETE /identity/v3/users/9296e886f8cf42aeaf7d3bba4f0e70bb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:48:51.857443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:51.858038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:51.858038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:51.877243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:48:51.878569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:51.937691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1122/2238] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:48:51.948081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:51.949577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:52.010066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443efe-12a0-4fe0-94ee-3e0638fb2103 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:52.013738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b8346-3f1c-4728-8996-d0393fedf042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:52.014349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b8346-3f1c-4728-8996-d0393fedf042 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:52.014620 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b8346-3f1c-4728-8996-d0393fedf042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:52.014930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b8346-3f1c-4728-8996-d0393fedf042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:52.025618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b8346-3f1c-4728-8996-d0393fedf042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:52.027132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b8346-3f1c-4728-8996-d0393fedf042 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:52.037096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1117/2239] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:48:52.348403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:48:52.349842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:48:52.363144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e9214-b9f7-4c54-a586-501e2ec132ad None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:48:52.368795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:48:52.369475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:48:52.369716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:48:52.369986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] PATH_INFO: `/v3/projects/bd2c477bc5254a79aa66ff975f8a02ad` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:48:52.371113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2c477bc5254a79aa66ff975f8a02ad)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:48:52.374105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:48:52.418637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:48:52.430188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2c477bc5254a79aa66ff975f8a02ad', 'request_id': 'req-a95b083f-d1d3-44c3-900f-755749ad06d4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:52.439329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2c477bc5254a79aa66ff975f8a02ad', 'request_id': 'req-a95b083f-d1d3-44c3-900f-755749ad06d4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:52.446722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2c477bc5254a79aa66ff975f8a02ad', 'request_id': 'req-a95b083f-d1d3-44c3-900f-755749ad06d4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:52.458300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] The token cache is being invalidate because project bd2c477bc5254a79aa66ff975f8a02ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:48:52.458585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:52.459303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a95b083f-d1d3-44c3-900f-755749ad06d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:48:52.460893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1123/2240] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:48:52 2022] DELETE /identity/v3/projects/bd2c477bc5254a79aa66ff975f8a02ad => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:49:02.032678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:02.455517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:09.430881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:09.431603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:09.431603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:09.450587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] MFA Rules not processed for user `f417a94fed274bfbb88d23677a4877a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:49:09.451815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:09.505372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1118/2241] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:49:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:49:19.498936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=25 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:32.821738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:32.822262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:32.822262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:32.842030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] MFA Rules not processed for user `6974a4c8fffd4bc2933e754770cb912e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:49:32.843315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:32.896792 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1124/2242] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:49:42.890180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=25 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:44.120360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:44.122098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:44.195024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:44.196624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:44.201547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ae370-1b60-48e9-b179-1bb736127255 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:44.206133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:44.206877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:44.207100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:44.207391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:44.209322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:44.210203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2133e-5935-4c51-a612-b72752f753b2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:44.213870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:44.214482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:44.214756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:44.215024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:44.217740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:44.299193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:44.299766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:44.300759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685047e1-fcc1-4645-925c-5f45c77bcab4 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:44.301228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:44.311938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1125/2243] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:49:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:49:44.313898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1119/2244] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:49:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:49:46.611618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:46.612282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:46.612282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:46.657350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:49:46.659150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:46.733088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1126/2245] 10.222.0.55 () {58 vars in 967 bytes} [Sat Sep 24 19:49:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:49:46.804714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:46.807137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:46.891839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6848c7-d8da-4767-a5cf-cfe8c16035da service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:46.898339 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:46.899257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:46.899815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:46.900116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:46.912363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:46.914466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:46.926023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1120/2246] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:49:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:49:54.444326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:54.445821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:54.527871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe764bf9-b2a9-4819-b678-d6b012948ecd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:54.535955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:54.536974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:54.537352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:54.537896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] PATH_INFO: `/v3/users/47773b38da48458fa356f34d259d3478` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:54.539632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] RBAC: Authorizing `identity:delete_user(user_id=47773b38da48458fa356f34d259d3478)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:54.543129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:54.588568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:54.603973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47773b38da48458fa356f34d259d3478'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.618653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47773b38da48458fa356f34d259d3478'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.625870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47773b38da48458fa356f34d259d3478'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.626610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47773b38da48458fa356f34d259d3478'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.633666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47773b38da48458fa356f34d259d3478'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.639969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1127/2247] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:49:54 2022] DELETE /identity/v3/users/47773b38da48458fa356f34d259d3478 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:49:54.823203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:54.836624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a46353-eb3e-4744-9713-808a8fd38529 service swift] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:54.873824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:54.907470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:54.908381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:54.908854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:54.909134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] PATH_INFO: `/v3/projects/8cb38d1aae344b868976df799b74b0f5` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:54.910462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb38d1aae344b868976df799b74b0f5)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:54.912489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:54.970529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb38d1aae344b868976df799b74b0f5', 'request_id': 'req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.979747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb38d1aae344b868976df799b74b0f5', 'request_id': 'req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:54.986764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb38d1aae344b868976df799b74b0f5', 'request_id': 'req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.002248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] The token cache is being invalidate because project 8cb38d1aae344b868976df799b74b0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:49:55.002538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.003202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.004810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1121/2248] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:49:54 2022] DELETE /identity/v3/projects/8cb38d1aae344b868976df799b74b0f5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:49:55.008969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:55.011160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:55.025364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ad33-cfae-45c4-898d-1364b3d5531b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:55.088825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:55.089461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:55.089739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:55.090006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] PATH_INFO: `/v3/users/b590de08f34d4265a78860006a804f1c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:55.091136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b590de08f34d4265a78860006a804f1c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:55.093698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:55.142417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b590de08f34d4265a78860006a804f1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.157776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b590de08f34d4265a78860006a804f1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.166000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b590de08f34d4265a78860006a804f1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.166715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b590de08f34d4265a78860006a804f1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.174232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b590de08f34d4265a78860006a804f1c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.180484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1128/2249] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:49:55 2022] DELETE /identity/v3/users/b590de08f34d4265a78860006a804f1c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:49:55.365803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:55.367595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:55.381663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b69b4-1d35-4dc0-9706-485ef3b2c471 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:55.447271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:55.448508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:55.448508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:55.448508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] PATH_INFO: `/v3/projects/4abf7cd541ca4a49961d89fe4ae24071` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:55.449639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4abf7cd541ca4a49961d89fe4ae24071)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:55.451626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:55.509346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4abf7cd541ca4a49961d89fe4ae24071', 'request_id': 'req-03e83e63-0857-4898-8386-84b457d87b38'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.519063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4abf7cd541ca4a49961d89fe4ae24071', 'request_id': 'req-03e83e63-0857-4898-8386-84b457d87b38'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.525465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:55.527118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4abf7cd541ca4a49961d89fe4ae24071', 'request_id': 'req-03e83e63-0857-4898-8386-84b457d87b38'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.545210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] The token cache is being invalidate because project 4abf7cd541ca4a49961d89fe4ae24071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:49:55.545662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.547271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:49:55.549373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1122/2250] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:49:55 2022] DELETE /identity/v3/projects/4abf7cd541ca4a49961d89fe4ae24071 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:49:56.409730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:56.411203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:56.445344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:49:56.475094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f721c54-05e5-49e0-b86b-320dc087a307 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:56.523263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:56.523806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:56.524080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:56.524345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:56.526097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:56.583909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:56.585356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:56.595257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1129/2251] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:49:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:49:59.811722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:49:59.813538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:59.889187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e83e63-0857-4898-8386-84b457d87b38 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:49:59.895382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:49:59.896092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:49:59.896377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:49:59.896668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:49:59.898450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:49:59.958799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:49:59.960421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e66ea-358b-4b0b-90b0-20b8eeee978e service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:49:59.969936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1123/2252] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:49:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:06.545530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:06.546991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:06.558085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c9477-0919-4206-b4e7-b28f543bdf92 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:06.561817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:06.562421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:06.562728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:06.563000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:06.564827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:06.629607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:06.631108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:06.642280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1130/2253] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:09.393574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:09.394136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:09.394227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:09.419517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] MFA Rules not processed for user `be2ad8e7eefe447ca1374d49696b1318`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:09.422181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:09.429490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1124/2254] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:09.769446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:09.771371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:09.822594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:09.841449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda166-c979-4439-96e4-661e4443b664 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:09.845432 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:09.846058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:09.846346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:09.846610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:09.848500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:09.860471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:09.861273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76593a-b7e2-42f5-89fc-2c873269d7fc service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:09.870882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1131/2255] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:50:10.268495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:10.270539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:10.351516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad172ab7-d17a-4479-8565-1c07e730386e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:10.356654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:10.357376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:10.357771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:10.358180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:10.360909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:10.373351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:10.375011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:10.387287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1125/2256] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:16.682636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:16.683228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:16.683801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:16.704789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] MFA Rules not processed for user `444719e34bf2457b9a58e182ef7fbd79`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:16.706004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:16.713008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1132/2257] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:17.055947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:17.057487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:17.070952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df917dac-5e1b-4818-adcb-326e0d33dc26 service placement] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:17.075195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:17.075911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:17.076248 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:17.076554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:17.078479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:17.089246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:17.090613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:17.094370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:17.102227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1126/2258] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:18.139093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:18.149121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f905-7e17-44f8-9905-62c6c8f62798 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:18.152620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:18.153210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:18.153486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:18.153753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:18.155449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:18.166523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:18.167968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:18.178365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1133/2259] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:21.178265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:21.189252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3ad9-831d-42e4-a250-dd5037f22e20 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:21.193538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:21.194310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:21.194659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:21.195002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:21.196943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:21.289632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:21.291093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:21.303337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1127/2260] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:21.573903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:21.588990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b9ea-4ab9-4d17-8db9-ffefb4b3e559 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:21.598784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:21.599597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:21.600120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:21.600595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:21.603629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:21.617447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:21.619078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:21.632401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1134/2261] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:25.151255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:25.155345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:25.185115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e896dff-37b4-452c-9823-6e8c94ee4bc5 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:25.192972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:25.193997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:25.195155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:25.195275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] PATH_INFO: `/v3/users/f417a94fed274bfbb88d23677a4877a7` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:25.196654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f417a94fed274bfbb88d23677a4877a7)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:25.200695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:25.268611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f417a94fed274bfbb88d23677a4877a7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.280039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f417a94fed274bfbb88d23677a4877a7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.281264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f417a94fed274bfbb88d23677a4877a7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.282202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f417a94fed274bfbb88d23677a4877a7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.293773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4d718132-4484-48aa-bcc8-4fa33d5795dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f417a94fed274bfbb88d23677a4877a7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.316207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1128/2262] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:25 2022] DELETE /identity/v3/users/f417a94fed274bfbb88d23677a4877a7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:25.534916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:25.548667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da3d71-062e-47db-8372-8458f2697069 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:25.619129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:25.619865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:25.620243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:25.620599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] PATH_INFO: `/v3/projects/0d671773b58240b1988f7c56ec6753b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:25.621876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d671773b58240b1988f7c56ec6753b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:25.623929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:25.681406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d671773b58240b1988f7c56ec6753b5', 'request_id': 'req-97729926-7aed-43ea-aaa9-a1c4efdd5e28'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.691883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d671773b58240b1988f7c56ec6753b5', 'request_id': 'req-97729926-7aed-43ea-aaa9-a1c4efdd5e28'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.698419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d671773b58240b1988f7c56ec6753b5', 'request_id': 'req-97729926-7aed-43ea-aaa9-a1c4efdd5e28'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.715033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] The token cache is being invalidate because project 0d671773b58240b1988f7c56ec6753b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:25.716004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.716955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-97729926-7aed-43ea-aaa9-a1c4efdd5e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:25.719284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1135/2263] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:25 2022] DELETE /identity/v3/projects/0d671773b58240b1988f7c56ec6753b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:25.724340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:25.724643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:25.724913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:25.747842 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:25.749081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:25.828381 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1129/2264] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:25.836623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:25.837021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:25.837361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:25.863512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:25.864752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:25.873058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1136/2265] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:25.877818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:25.884821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:25.889079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126ecf3-22ee-4c73-b063-fb633686922e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:25.896184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:25.896860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:25.897391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:25.898956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:25.898956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:50:25.898956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:25.900709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:25.915155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1130/2266] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:50:25.920020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:25.929808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d50e0-ce7e-458d-a034-0c0822b820eb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:25.934833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:25.935451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:25.935764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:25.936028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:25.936686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:25.939163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:25.970133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1137/2267] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:25 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:25.974390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:25.984920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba098e-44fa-4834-be1d-94a0bb775f5b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.048322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.049073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.049475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.049745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.050498 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.052866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.081807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1131/2268] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:25 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:26.086011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.096900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f85d8-f326-483a-bc0d-e503b435bc83 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.101798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.102427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.102727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.103168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.103860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.105467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.108131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:26.119563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1138/2269] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:26.123496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.134578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d96aa-9a1f-46ab-8f4b-f7b9d68d3192 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.139366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05402-0c1b-4963-b3e7-4b85417f947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.140014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05402-0c1b-4963-b3e7-4b85417f947a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.140924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05402-0c1b-4963-b3e7-4b85417f947a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.141297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05402-0c1b-4963-b3e7-4b85417f947a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.141954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee05402-0c1b-4963-b3e7-4b85417f947a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.144028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee05402-0c1b-4963-b3e7-4b85417f947a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.156358 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1132/2270] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:50:26.171265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.181701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16c944-56e8-45a8-8303-dae778c60c20 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.186396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.187612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.187890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.188159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] PATH_INFO: `/v3/projects/28d1564988414b0ab9f766ce310c4918/users/f1e70105405d4bb08dea7091b8a2c468/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.205352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d1564988414b0ab9f766ce310c4918, user_id=f1e70105405d4bb08dea7091b8a2c468, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.208364 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.223657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1139/2271] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:26 2022] PUT /identity/v3/projects/28d1564988414b0ab9f766ce310c4918/users/f1e70105405d4bb08dea7091b8a2c468/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:26.236091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.236378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.236653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.267930 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] MFA Rules not processed for user `f1e70105405d4bb08dea7091b8a2c468`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:26.269587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:26.352406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1133/2272] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:26.353559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.440312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b39a-a2e6-4ea1-a906-1d0677b3f9d1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.459864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.460490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.460766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.461614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.461828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.463928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.508666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1140/2273] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:26 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:26.514736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.539969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487e05-7633-4b1e-9629-1aa34ffd06e0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.612394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.613074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.613347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.613616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.614260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.616343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.626231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:26.643710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1134/2274] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:26 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:26.647662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.662299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d8234-1cd3-4838-8173-79a87d68a3d1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.667406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.668433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.668721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.668988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.669886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.672086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.683673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1141/2275] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:26.688531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.698521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab7c9b-7604-40c4-863a-941a63236bb8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.709122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.709813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.710090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.710360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] PATH_INFO: `/v3/projects/3c35884412b04ceea4fa20527c093f19/users/13097e7d7ffb45199eb4ad29f0e94a1a/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.736460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c35884412b04ceea4fa20527c093f19, user_id=13097e7d7ffb45199eb4ad29f0e94a1a, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.741658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.758297 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1135/2276] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:26 2022] PUT /identity/v3/projects/3c35884412b04ceea4fa20527c093f19/users/13097e7d7ffb45199eb4ad29f0e94a1a/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:26.762730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.772971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58d2d7-da15-4c9d-988d-7df23f3aaac2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.831506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a488d7d-71a2-4d2e-8ae7-f5cef6fcf724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.832030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a488d7d-71a2-4d2e-8ae7-f5cef6fcf724 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.832310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a488d7d-71a2-4d2e-8ae7-f5cef6fcf724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.832580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a488d7d-71a2-4d2e-8ae7-f5cef6fcf724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.833219 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a488d7d-71a2-4d2e-8ae7-f5cef6fcf724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.834826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a488d7d-71a2-4d2e-8ae7-f5cef6fcf724 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.846852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1142/2277] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:50:26.851139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:26.864203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d865a-94b0-4456-b457-e08d0fdcc3b7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:26.870225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:26.870997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.871161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.871508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] PATH_INFO: `/v3/projects/3c35884412b04ceea4fa20527c093f19/users/13097e7d7ffb45199eb4ad29f0e94a1a/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.873483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c35884412b04ceea4fa20527c093f19, user_id=13097e7d7ffb45199eb4ad29f0e94a1a, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:26.876547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:26.891478 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1136/2278] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:26 2022] PUT /identity/v3/projects/3c35884412b04ceea4fa20527c093f19/users/13097e7d7ffb45199eb4ad29f0e94a1a/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:26.905275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:26.906883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:26.907059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:26.944868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] MFA Rules not processed for user `13097e7d7ffb45199eb4ad29f0e94a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:26.946064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:26.986752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:27.032646 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1143/2279] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:27.043867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:27.046019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:27.124357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae991fa-9ae0-44e8-9ee9-deee72b57ee2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:27.130259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58627435-be5c-4349-9c0d-810391405c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:27.130882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58627435-be5c-4349-9c0d-810391405c0f service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:27.131161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58627435-be5c-4349-9c0d-810391405c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:27.131488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58627435-be5c-4349-9c0d-810391405c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:27.142246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58627435-be5c-4349-9c0d-810391405c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:27.143772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58627435-be5c-4349-9c0d-810391405c0f service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:27.154041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1137/2280] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:27.185466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:27.250319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21e914-e8e5-4c7d-b457-739e87c3acf6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:27.254991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:27.255659 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:27.255940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:27.256209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] PATH_INFO: `/v3/users/f1e70105405d4bb08dea7091b8a2c468` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:27.257367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e70105405d4bb08dea7091b8a2c468)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:27.259347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:27.313684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e70105405d4bb08dea7091b8a2c468'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:27.325668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e70105405d4bb08dea7091b8a2c468'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:27.334019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e70105405d4bb08dea7091b8a2c468'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:27.334643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e70105405d4bb08dea7091b8a2c468'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:27.341760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e70105405d4bb08dea7091b8a2c468'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:27.348228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1144/2281] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:27 2022] DELETE /identity/v3/users/f1e70105405d4bb08dea7091b8a2c468 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:27.352889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:27.353410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:27.353508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:27.372337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:27.373539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:27.435109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1138/2282] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:27.445056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:27.446488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:27.502587 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:27.506977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27111c77-239e-4cd7-8596-a3385564f7bc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:27.512099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:27.512902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:27.513377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:27.513803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:27.524456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:27.525898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:27.537335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1145/2283] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:27.901327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:27.902779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:27.922490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55d9c7-61b1-4342-abd8-a8c8fbd8fba0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:27.929019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:27.938816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:27.939230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:27.939604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] PATH_INFO: `/v3/projects/28d1564988414b0ab9f766ce310c4918` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:27.941105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d1564988414b0ab9f766ce310c4918)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:27.943586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:27.997736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d1564988414b0ab9f766ce310c4918', 'request_id': 'req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.000308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:28.008140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d1564988414b0ab9f766ce310c4918', 'request_id': 'req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.018477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d1564988414b0ab9f766ce310c4918', 'request_id': 'req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.030007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] The token cache is being invalidate because project 28d1564988414b0ab9f766ce310c4918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:28.030296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.030989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.032531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1139/2284] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:27 2022] DELETE /identity/v3/projects/28d1564988414b0ab9f766ce310c4918 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:28.036583 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:28.047213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410d650-516c-412a-baf8-4b868aa41088 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:28.132141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:28.132776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:28.133051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:28.133347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] PATH_INFO: `/v3/users/13097e7d7ffb45199eb4ad29f0e94a1a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:28.134505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13097e7d7ffb45199eb4ad29f0e94a1a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:28.136498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:28.215109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13097e7d7ffb45199eb4ad29f0e94a1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.232798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13097e7d7ffb45199eb4ad29f0e94a1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.246521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13097e7d7ffb45199eb4ad29f0e94a1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.246910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13097e7d7ffb45199eb4ad29f0e94a1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.255370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f6ac5d7c-bc5b-45d1-9ba9-cb20abeb9c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13097e7d7ffb45199eb4ad29f0e94a1a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.272331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1146/2285] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:28 2022] DELETE /identity/v3/users/13097e7d7ffb45199eb4ad29f0e94a1a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:28.728271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:28.730216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:28.752143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3068-72c1-44f1-8be3-4ee9b4abdbc3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:28.836012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:28.836666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:28.836950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:28.837228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] PATH_INFO: `/v3/projects/3c35884412b04ceea4fa20527c093f19` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:28.838424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c35884412b04ceea4fa20527c093f19)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:28.840548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:28.896604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c35884412b04ceea4fa20527c093f19', 'request_id': 'req-93268aa0-f82d-4988-b0ae-520b33d33c76'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.909249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c35884412b04ceea4fa20527c093f19', 'request_id': 'req-93268aa0-f82d-4988-b0ae-520b33d33c76'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.920921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c35884412b04ceea4fa20527c093f19', 'request_id': 'req-93268aa0-f82d-4988-b0ae-520b33d33c76'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.934956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] The token cache is being invalidate because project 3c35884412b04ceea4fa20527c093f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:28.935250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.935957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-93268aa0-f82d-4988-b0ae-520b33d33c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:28.938556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1140/2286] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:28 2022] DELETE /identity/v3/projects/3c35884412b04ceea4fa20527c093f19 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:28.943851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:28.944064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:28.944371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:28.966113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:28.967512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:29.035729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1147/2287] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:29.043399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.043797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.044379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.066987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:29.068257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:29.076544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1141/2288] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:29.081029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.091137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ad00c-6831-4721-b1f8-38f68e54eae1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.097058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.097677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.097955 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.098230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.098917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:50:29.099216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.100914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.108647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:29.115069 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1148/2289] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:50:29.119892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.130141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050708f-b7d1-4899-9055-e94b8eb743bb None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.135082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.135727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.136070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.136344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.136996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.138951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.175650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1142/2290] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:29 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:29.179884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.192531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a27b0-ef94-4fd4-b8f1-7552e16a0ca6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.261403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.262140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.262503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.262778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.263525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.265591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.295661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1149/2291] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:29 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:29.297616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.311746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809270b-13bf-480e-a610-5c7b8100af98 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.318916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.319584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.319913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.320803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.321479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.323195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.348230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1143/2292] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:29.349565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.376537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972176-9ae3-4e3a-b537-89a2651933ef admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.382342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29accb-05e4-4da2-ba4c-fdbc04f3b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.382972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29accb-05e4-4da2-ba4c-fdbc04f3b1db admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.383255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29accb-05e4-4da2-ba4c-fdbc04f3b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.383572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29accb-05e4-4da2-ba4c-fdbc04f3b1db admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.384199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29accb-05e4-4da2-ba4c-fdbc04f3b1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.385816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29accb-05e4-4da2-ba4c-fdbc04f3b1db admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.400490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1150/2293] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:50:29.402546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.413905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b89d77-40ca-4bf7-abb9-69b6f9077a0a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.426315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.426978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.427252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.427550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] PATH_INFO: `/v3/projects/6868c5355beb48cfb6f9b69e71732b91/users/74b3bfbbe31d4d2ebb48476c56f401fc/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.445923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6868c5355beb48cfb6f9b69e71732b91, user_id=74b3bfbbe31d4d2ebb48476c56f401fc, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.448667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.471152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1144/2294] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:29 2022] PUT /identity/v3/projects/6868c5355beb48cfb6f9b69e71732b91/users/74b3bfbbe31d4d2ebb48476c56f401fc/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:29.485582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.486255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.486680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.527357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] MFA Rules not processed for user `74b3bfbbe31d4d2ebb48476c56f401fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:29.528684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:29.630858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1151/2295] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:29.632350 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.639722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:29.736801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98a738-ed95-425f-acef-d6d0adf78301 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.750022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.752519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.753187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.753679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.756521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.758745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.798241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1145/2296] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:29 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:29.800655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.813137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8fe64-ed45-4c98-bdd7-faca3becf5a9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.895984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.897398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.897733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.898338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.899576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.902660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.922242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:29.946468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1152/2297] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:29 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:50:29.947817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:29.964242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883e1b-fbec-44a3-808e-62de24e14f56 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:29.971482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:29.972421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:29.972421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:29.972654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:29.973589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:29.975222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:29.990670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1146/2298] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:50:29.993199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.009128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540aa69-8c39-4872-9278-3ee906fb77d3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:30.018375 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:30.019601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.020754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.021041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] PATH_INFO: `/v3/projects/592640cacbfd41c988bdfb052551e66d/users/5a72d35dc5c74a528914f2fd5518934e/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.060887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592640cacbfd41c988bdfb052551e66d, user_id=5a72d35dc5c74a528914f2fd5518934e, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:30.064221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:30.085430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1153/2299] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:29 2022] PUT /identity/v3/projects/592640cacbfd41c988bdfb052551e66d/users/5a72d35dc5c74a528914f2fd5518934e/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:30.093296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.118458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c68e91-246f-407c-a4cb-ff5f2dd706c6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:30.194694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766a94-c26d-48a5-a0e1-a1ba9041a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:30.196285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766a94-c26d-48a5-a0e1-a1ba9041a6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.196285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766a94-c26d-48a5-a0e1-a1ba9041a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.196285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766a94-c26d-48a5-a0e1-a1ba9041a6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.196877 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67766a94-c26d-48a5-a0e1-a1ba9041a6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:30.198633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67766a94-c26d-48a5-a0e1-a1ba9041a6b2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:30.214552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1147/2300] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:30.216094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.234218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7a17-e1f1-459b-ab93-b04244c2ed21 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:30.239574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:30.240263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.240547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.240878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] PATH_INFO: `/v3/projects/592640cacbfd41c988bdfb052551e66d/users/5a72d35dc5c74a528914f2fd5518934e/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.242878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592640cacbfd41c988bdfb052551e66d, user_id=5a72d35dc5c74a528914f2fd5518934e, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:30.245654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:30.263605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1154/2301] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:30 2022] PUT /identity/v3/projects/592640cacbfd41c988bdfb052551e66d/users/5a72d35dc5c74a528914f2fd5518934e/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:30.269788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.270123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.270332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.303585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] MFA Rules not processed for user `5a72d35dc5c74a528914f2fd5518934e`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:30.304813 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:30.386923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:30.392263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1148/2302] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:30.409301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.412427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:30.481347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee4b73-9053-4eac-9622-241bb81329e7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:30.486030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:30.486030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.487729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.488284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.503090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:30.504504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:30.516043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1155/2303] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:30.605203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.684342 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03402b-8770-42dd-bc79-f6a6e57a9c47 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:30.690486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:30.691137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.691445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.691734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] PATH_INFO: `/v3/users/74b3bfbbe31d4d2ebb48476c56f401fc` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.695356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b3bfbbe31d4d2ebb48476c56f401fc)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:30.697241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:30.771972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b3bfbbe31d4d2ebb48476c56f401fc'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:30.787450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b3bfbbe31d4d2ebb48476c56f401fc'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:30.788173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b3bfbbe31d4d2ebb48476c56f401fc'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:30.788856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b3bfbbe31d4d2ebb48476c56f401fc'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:30.799643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-690d788c-12fd-4b70-a578-4710637b6d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b3bfbbe31d4d2ebb48476c56f401fc'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:30.817503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.819504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:30.824882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1149/2304] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:30 2022] DELETE /identity/v3/users/74b3bfbbe31d4d2ebb48476c56f401fc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:30.826395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.826695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.826991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.852918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:30.854679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:30.896095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f2e5-a601-43af-8976-aa393734cc63 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:30.931259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1150/2305] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:30.942838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:30.944696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:30.958529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:30.972114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:30.977987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:30.978637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:30.978961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:30.979257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:30.981122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:31.015117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc712-c10e-4e09-8341-29ad607465e4 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:31.019317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:31.019891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:31.020165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:31.020546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:31.030665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:31.032168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:31.047680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1151/2306] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:31.056787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:31.058739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:31.069994 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1156/2307] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:31.438261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:31.440762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:31.473286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4d0ca-1e4c-4277-99e2-10321899d5b8 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:31.498362 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:31.499773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:31.500338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:31.500757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] PATH_INFO: `/v3/projects/6868c5355beb48cfb6f9b69e71732b91` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:31.502212 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6868c5355beb48cfb6f9b69e71732b91)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:31.504437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:31.612939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6868c5355beb48cfb6f9b69e71732b91', 'request_id': 'req-138f9d54-d599-4f18-87af-79f4ac6ba77f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.642486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6868c5355beb48cfb6f9b69e71732b91', 'request_id': 'req-138f9d54-d599-4f18-87af-79f4ac6ba77f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.652228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6868c5355beb48cfb6f9b69e71732b91', 'request_id': 'req-138f9d54-d599-4f18-87af-79f4ac6ba77f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.679815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] The token cache is being invalidate because project 6868c5355beb48cfb6f9b69e71732b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:31.680232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.681008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.682710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1152/2308] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:31 2022] DELETE /identity/v3/projects/6868c5355beb48cfb6f9b69e71732b91 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:31.686637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:31.698289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12befa20-4318-45b1-8edb-1c5cf6349763 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:31.772472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:31.773125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:31.773471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:31.773807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] PATH_INFO: `/v3/users/5a72d35dc5c74a528914f2fd5518934e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:31.775774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a72d35dc5c74a528914f2fd5518934e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:31.778598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:31.847492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a72d35dc5c74a528914f2fd5518934e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.864722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a72d35dc5c74a528914f2fd5518934e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.873764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a72d35dc5c74a528914f2fd5518934e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.875447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a72d35dc5c74a528914f2fd5518934e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.883767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfb2dba1-ff71-49d2-b9c2-18685f2f2e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a72d35dc5c74a528914f2fd5518934e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:31.895748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1157/2309] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:31 2022] DELETE /identity/v3/users/5a72d35dc5c74a528914f2fd5518934e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:32.399954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:32.401644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:32.418546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9d54-d599-4f18-87af-79f4ac6ba77f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:32.495654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:32.496426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.496768 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.497108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] PATH_INFO: `/v3/projects/592640cacbfd41c988bdfb052551e66d` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.498875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592640cacbfd41c988bdfb052551e66d)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:32.501582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:32.557749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:32.559336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592640cacbfd41c988bdfb052551e66d', 'request_id': 'req-4598259d-3267-4872-a751-55e62609dac0'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:32.569986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592640cacbfd41c988bdfb052551e66d', 'request_id': 'req-4598259d-3267-4872-a751-55e62609dac0'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:32.579227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592640cacbfd41c988bdfb052551e66d', 'request_id': 'req-4598259d-3267-4872-a751-55e62609dac0'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:32.591871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] The token cache is being invalidate because project 592640cacbfd41c988bdfb052551e66d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:32.592155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:32.593188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4598259d-3267-4872-a751-55e62609dac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:32.595064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1153/2310] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:32 2022] DELETE /identity/v3/projects/592640cacbfd41c988bdfb052551e66d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:32.600886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.601397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.602136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.631474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:32.631474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:32.643573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:32.697168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1158/2311] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:32.705119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.705422 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.705665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.728438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:32.729643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:32.740695 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1154/2312] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:32.744911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:32.756021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c6660-0226-486a-b452-c180e665d11a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:32.760675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:32.761286 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.761567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.761840 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.762487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:50:32.762790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:32.764418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:32.776308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1159/2313] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:50:32.780647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:32.790111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20baa4-f8a3-4030-a9f5-fb39ada3e9e3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:32.795144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:32.795778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.796572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.796849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.797480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:32.799390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:32.829055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1155/2314] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:32 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:32.833270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:32.844924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f811c3-614c-43ae-9dbf-c8d68f52f928 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:32.901742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:32.902373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.902631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.902903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.903585 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:32.905996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:32.929663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1160/2315] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:32 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:32.934029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:32.945216 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea30ef8-e755-4e6b-8d08-42106a491318 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:32.950921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:32.951591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.951896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.952194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.952871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:32.954514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:32.970098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1156/2316] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:32.974492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:32.987258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec00a-acdf-4edb-af7e-ac4ef3d0db26 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:32.992099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016d95c-c523-45d8-a079-0e3f787ff08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:32.992715 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016d95c-c523-45d8-a079-0e3f787ff08e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:32.992986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016d95c-c523-45d8-a079-0e3f787ff08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:32.993256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016d95c-c523-45d8-a079-0e3f787ff08e admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:32.993899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016d95c-c523-45d8-a079-0e3f787ff08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:32.995529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016d95c-c523-45d8-a079-0e3f787ff08e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.006909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1161/2317] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:33.011101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.022652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74fa9b-81b5-4273-b0e0-23dc69d0e993 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.027760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.028377 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.028699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.029013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] PATH_INFO: `/v3/projects/6655c5cd98d8420b8c88f68019b53ef3/users/67fe456280044e5197ff4373c111543b/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.048081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6655c5cd98d8420b8c88f68019b53ef3, user_id=67fe456280044e5197ff4373c111543b, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.050824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.067218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1157/2318] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:33 2022] PUT /identity/v3/projects/6655c5cd98d8420b8c88f68019b53ef3/users/67fe456280044e5197ff4373c111543b/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:33.078834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee65238-cbba-4215-963c-5e733768217c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.087796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee65238-cbba-4215-963c-5e733768217c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.088108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee65238-cbba-4215-963c-5e733768217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.130554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9ee65238-cbba-4215-963c-5e733768217c None None] MFA Rules not processed for user `67fe456280044e5197ff4373c111543b`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:33.131804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9ee65238-cbba-4215-963c-5e733768217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:33.199324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1162/2319] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:33.204832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.285953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc8d1d-0cd3-4413-b5b9-60b1a3087448 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.294825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.295539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.295849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.296371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.296983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.299194 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.328908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1158/2320] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:33 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:33.334108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee65238-cbba-4215-963c-5e733768217c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.346246 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee65238-cbba-4215-963c-5e733768217c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.355478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:33.423886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.424526 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.424808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.425082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.425741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.427747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.457011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1163/2321] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:33 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:50:33.458775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.470818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34503e-2364-460e-840f-8050a9ea6fcf admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.476112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.476494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.476770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.477038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.477686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.479318 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.493968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1159/2322] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:33.495004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.505449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36dd5c-1039-4416-978a-9d7412b91373 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.510038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.510689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.511102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.511386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] PATH_INFO: `/v3/projects/fe077daf950e4b3595730d0873bbe411/users/75242b080f9d405e978fbbb858afa740/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.527692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe077daf950e4b3595730d0873bbe411, user_id=75242b080f9d405e978fbbb858afa740, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.530382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.544632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1164/2323] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:33 2022] PUT /identity/v3/projects/fe077daf950e4b3595730d0873bbe411/users/75242b080f9d405e978fbbb858afa740/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:33.548670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.558736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b25f0-4112-41c1-876f-605694e97b86 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.613506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:33.628504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68a303-0308-498e-856a-88ff975ccb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.629113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68a303-0308-498e-856a-88ff975ccb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.629389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68a303-0308-498e-856a-88ff975ccb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.629657 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68a303-0308-498e-856a-88ff975ccb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.630299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68a303-0308-498e-856a-88ff975ccb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.631951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68a303-0308-498e-856a-88ff975ccb94 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.642693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1160/2324] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:50:33.646945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.661910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898299c-c68b-4a75-aa8f-b70302ab4195 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.667401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.668093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.668378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.668650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] PATH_INFO: `/v3/projects/fe077daf950e4b3595730d0873bbe411/users/75242b080f9d405e978fbbb858afa740/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.670820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe077daf950e4b3595730d0873bbe411, user_id=75242b080f9d405e978fbbb858afa740, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.673911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.684482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1165/2325] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:33 2022] PUT /identity/v3/projects/fe077daf950e4b3595730d0873bbe411/users/75242b080f9d405e978fbbb858afa740/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:33.695611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.695897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.696171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.726019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] MFA Rules not processed for user `75242b080f9d405e978fbbb858afa740`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:33.727272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:33.797867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1161/2326] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:33.809355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.811053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:33.869817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a2b1f-f4a6-47e4-ae97-9120844e6178 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.873374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.873973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.874881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.875124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:33.883977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:33.885403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:33.895222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1166/2327] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:33.967544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:33.977643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1fee1-8fbc-4757-a5ae-c6b2a9c318ca None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:33.981372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e71e9-0f7f-4921-a942-c3626408f55f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:33.981988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e71e9-0f7f-4921-a942-c3626408f55f service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:33.982272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e71e9-0f7f-4921-a942-c3626408f55f service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:33.982541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e71e9-0f7f-4921-a942-c3626408f55f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:34.057102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e71e9-0f7f-4921-a942-c3626408f55f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:34.058696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e71e9-0f7f-4921-a942-c3626408f55f service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:34.079848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1162/2328] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:34.361537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:34.431724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d96e78-70f3-4a22-997f-9694129d6420 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:34.436913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:34.437564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:34.437920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:34.438189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] PATH_INFO: `/v3/users/67fe456280044e5197ff4373c111543b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:34.439312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=67fe456280044e5197ff4373c111543b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:34.441426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:34.447782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:34.495522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67fe456280044e5197ff4373c111543b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:34.507940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67fe456280044e5197ff4373c111543b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:34.516137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67fe456280044e5197ff4373c111543b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:34.516773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67fe456280044e5197ff4373c111543b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:34.525779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67fe456280044e5197ff4373c111543b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:34.533193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1167/2329] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:34 2022] DELETE /identity/v3/users/67fe456280044e5197ff4373c111543b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:34.537742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:34.538452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:34.538452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:34.568488 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:34.569806 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:34.633743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1163/2330] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:34.647001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:34.648418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:34.716455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536d04-aecc-4f64-b070-dbb4b00bab3b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:34.721329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:34.722021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:34.722444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:34.723032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:34.734414 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:34.736516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:34.758610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1168/2331] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:50:35.047715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:35.049483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:35.079760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ceeba8-44d8-4539-99c6-2de326981052 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:35.085210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:35.085967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:35.086244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:35.086509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] PATH_INFO: `/v3/projects/6655c5cd98d8420b8c88f68019b53ef3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:35.087676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6655c5cd98d8420b8c88f68019b53ef3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:35.089595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:35.091262 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:35.146676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6655c5cd98d8420b8c88f68019b53ef3', 'request_id': 'req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.154597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6655c5cd98d8420b8c88f68019b53ef3', 'request_id': 'req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.161748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6655c5cd98d8420b8c88f68019b53ef3', 'request_id': 'req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.181459 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] The token cache is being invalidate because project 6655c5cd98d8420b8c88f68019b53ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:35.181780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.182489 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7a729a1b-6e72-44c6-b64a-8ca8aa5a08e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.184181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1164/2332] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:35 2022] DELETE /identity/v3/projects/6655c5cd98d8420b8c88f68019b53ef3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:35.189995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:35.201238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a4305-f980-4b13-a813-18632bdc15eb service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:35.259287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:35.259850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:35.260127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:35.260398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] PATH_INFO: `/v3/users/75242b080f9d405e978fbbb858afa740` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:35.261510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=75242b080f9d405e978fbbb858afa740)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:35.263500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:35.297083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:35.312237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75242b080f9d405e978fbbb858afa740'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.328473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75242b080f9d405e978fbbb858afa740'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.336675 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75242b080f9d405e978fbbb858afa740'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.337331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75242b080f9d405e978fbbb858afa740'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.347188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bb7bfa95-ce9d-440d-baea-d90539985eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75242b080f9d405e978fbbb858afa740'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:35.354486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1169/2333] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:35 2022] DELETE /identity/v3/users/75242b080f9d405e978fbbb858afa740 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:35.762180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc28cfc-223d-4436-91d2-60a37e3485a5 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:35.763110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc28cfc-223d-4436-91d2-60a37e3485a5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:35.763619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc28cfc-223d-4436-91d2-60a37e3485a5 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:35.765293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1165/2334] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:50:35.772444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:35.773829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:35.774343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:35.806318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] MFA Rules not processed for user `8702de7dc5fd4fdaa05449938d461de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:35.808642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:35.881618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1170/2335] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:35.906540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc28cfc-223d-4436-91d2-60a37e3485a5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:35.909464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bbc28cfc-223d-4436-91d2-60a37e3485a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:35.927627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc28cfc-223d-4436-91d2-60a37e3485a5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:35.932146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5671c-9519-4b6a-9735-3df622454fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:35.932966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5671c-9519-4b6a-9735-3df622454fbf service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:35.933404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5671c-9519-4b6a-9735-3df622454fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:35.934289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5671c-9519-4b6a-9735-3df622454fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:35.950981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5671c-9519-4b6a-9735-3df622454fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:35.952662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5671c-9519-4b6a-9735-3df622454fbf service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:35.961802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1166/2336] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:36.027204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.028769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.084805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03106b6f-0149-42a0-a7c7-ca37026191c6 None None] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.084805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03106b6f-0149-42a0-a7c7-ca37026191c6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.084805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03106b6f-0149-42a0-a7c7-ca37026191c6 None None] PATH_INFO: `/` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.084805 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1167/2337] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 19:50:36.086260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.086539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.086807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.108022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:36.109481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.122835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330dd28-1b9f-47db-a30d-48adbf234b49 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.128618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.129241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.129525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.129794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] PATH_INFO: `/v3/projects/fe077daf950e4b3595730d0873bbe411` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.130918 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe077daf950e4b3595730d0873bbe411)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.132929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.185430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe077daf950e4b3595730d0873bbe411', 'request_id': 'req-f4deb65f-9c11-4047-b317-f462bb8a9d26'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:36.194121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe077daf950e4b3595730d0873bbe411', 'request_id': 'req-f4deb65f-9c11-4047-b317-f462bb8a9d26'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:36.194983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1168/2338] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:36.197269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:36.202852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe077daf950e4b3595730d0873bbe411', 'request_id': 'req-f4deb65f-9c11-4047-b317-f462bb8a9d26'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:36.207306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.217238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d457-e2f9-4dc3-b284-4df20ddb283f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.225002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] The token cache is being invalidate because project fe077daf950e4b3595730d0873bbe411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:36.225495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:36.226522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4deb65f-9c11-4047-b317-f462bb8a9d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:36.229081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1171/2339] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:36 2022] DELETE /identity/v3/projects/fe077daf950e4b3595730d0873bbe411 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:36.235308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81a9b-5e7d-4de1-a37c-e3e756c017b7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.235612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81a9b-5e7d-4de1-a37c-e3e756c017b7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.235878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81a9b-5e7d-4de1-a37c-e3e756c017b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.256394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bdf81a9b-5e7d-4de1-a37c-e3e756c017b7 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:36.257614 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bdf81a9b-5e7d-4de1-a37c-e3e756c017b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.274275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.274802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.275552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.275831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.277589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.300258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:36.324299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1172/2340] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:36.331717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.332005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.332275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.351665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:36.352865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.353240 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.354683 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.361423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1173/2341] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:36.365461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.367168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1169/2342] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:50:36.375029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f39c88-dd5c-458e-8598-26fc8ee599ae None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.379857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.380457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.380739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.381009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.381647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:50:36.381946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.383556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.395277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1174/2343] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:50:36.399809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.410587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78617408-7674-4482-95ff-8bd7e7e4db0e service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.415312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.415949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.416223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.416494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.417123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.419034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.454192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1170/2344] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:36 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:36.458466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.469445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762042d-0581-4853-8206-607c2b71b906 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.538394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.539692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.540117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.540393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.541057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.543092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.567327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1175/2345] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:36 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:50:36.571493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.589456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1f1ab-254a-4ed8-bfca-a2dcfacaeaf5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.595148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.596069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.596509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.596824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.597412 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.599071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.612242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1171/2346] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:36.616898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.631125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98524420-32fd-4ce7-8ef9-c68d6315257d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.638996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec455c-2fd0-4298-8afc-f21d0fb2e5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.640047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec455c-2fd0-4298-8afc-f21d0fb2e5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.640511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec455c-2fd0-4298-8afc-f21d0fb2e5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.640962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec455c-2fd0-4298-8afc-f21d0fb2e5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.642053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec455c-2fd0-4298-8afc-f21d0fb2e5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.644952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec455c-2fd0-4298-8afc-f21d0fb2e5f2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.673140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1176/2347] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:50:36.677340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.687706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbf95d-4b25-45ed-afd4-009997e36786 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.692977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.693655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.693936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.694214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] PATH_INFO: `/v3/projects/b10beca4c2654152aaca02573deaefb7/users/acd4bcfd17624f10bde5184569e95661/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.713287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10beca4c2654152aaca02573deaefb7, user_id=acd4bcfd17624f10bde5184569e95661, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.716034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.730124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1172/2348] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:36 2022] PUT /identity/v3/projects/b10beca4c2654152aaca02573deaefb7/users/acd4bcfd17624f10bde5184569e95661/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:36.742190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.742472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.742741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.773824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] MFA Rules not processed for user `acd4bcfd17624f10bde5184569e95661`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:36.775107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.809922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:36.845405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1177/2349] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:36.857366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:36.859729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:36.918031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5de58-119a-4c35-bea6-5930b63217fe admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:36.923322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6204d-85dd-4250-9499-3d474ef2d0f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:36.923977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6204d-85dd-4250-9499-3d474ef2d0f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:36.924309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6204d-85dd-4250-9499-3d474ef2d0f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:36.924519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6204d-85dd-4250-9499-3d474ef2d0f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:36.942612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6204d-85dd-4250-9499-3d474ef2d0f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:36.945347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6204d-85dd-4250-9499-3d474ef2d0f3 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:36.972274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1173/2350] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:37.004016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:37.091119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1552b-0507-473c-bd52-d020eacc30d3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:37.096969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:37.097620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.097893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.098188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] PATH_INFO: `/v3/users/acd4bcfd17624f10bde5184569e95661` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.099462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd4bcfd17624f10bde5184569e95661)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:37.102179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:37.175764 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd4bcfd17624f10bde5184569e95661'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.192194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd4bcfd17624f10bde5184569e95661'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.201016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd4bcfd17624f10bde5184569e95661'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.202063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd4bcfd17624f10bde5184569e95661'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.212397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd4bcfd17624f10bde5184569e95661'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.223263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1178/2351] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:37 2022] DELETE /identity/v3/users/acd4bcfd17624f10bde5184569e95661 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:50:37.225340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.225629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.225902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.246832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:37.248062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:37.300664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:37.314143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1174/2352] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:37.324063 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:37.325594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:37.383471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba3d9-efba-440d-acfc-5263eebec5f0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:37.387601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3ea79-71b6-49cd-a302-aa1844164f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:37.388228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ea79-71b6-49cd-a302-aa1844164f25 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.388508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ea79-71b6-49cd-a302-aa1844164f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.388771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ea79-71b6-49cd-a302-aa1844164f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.400056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3ea79-71b6-49cd-a302-aa1844164f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:37.401503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3ea79-71b6-49cd-a302-aa1844164f25 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:37.405557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:37.412932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1179/2353] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:37.703800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:37.705581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:37.727084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e0e1-49f7-4771-bafe-583d2a33dabd None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:37.731964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:37.732592 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.732901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.733142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] PATH_INFO: `/v3/projects/b10beca4c2654152aaca02573deaefb7` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.735110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10beca4c2654152aaca02573deaefb7)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:37.737158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:37.786924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10beca4c2654152aaca02573deaefb7', 'request_id': 'req-b5877b37-b8e5-4005-811d-e6d45dd81dc9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.795628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10beca4c2654152aaca02573deaefb7', 'request_id': 'req-b5877b37-b8e5-4005-811d-e6d45dd81dc9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.803122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10beca4c2654152aaca02573deaefb7', 'request_id': 'req-b5877b37-b8e5-4005-811d-e6d45dd81dc9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.815324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] The token cache is being invalidate because project b10beca4c2654152aaca02573deaefb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:37.815543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.816181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b5877b37-b8e5-4005-811d-e6d45dd81dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:37.817667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1175/2354] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:37 2022] DELETE /identity/v3/projects/b10beca4c2654152aaca02573deaefb7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:37.822602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.822880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.823159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.846814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:37.848059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:37.914374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1180/2355] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:37.922475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.923030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.923030 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.945024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:37.946269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:37.955044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1176/2356] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:37.959396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:37.969969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84947e-cda0-4eb2-8690-700fa8932ea8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:37.974749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:37.975402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:37.975711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:37.975998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:37.976643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:50:37.976943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:37.978534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:37.990654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1181/2357] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:50:37.994903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.004403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd72fd-3c4c-404c-94d6-9eaf562f4fdb None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.009666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.010328 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.010614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.010882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.011540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.013455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.044816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1177/2358] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:37 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:38.049506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.062254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a2bf0-35f6-4336-8c55-707cb0e068e0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.132081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.132686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.132966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.133238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.133886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.135862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.157921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1182/2359] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:38 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:38.166325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.177122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d22a58-1139-41f8-a6a7-c415b7f8e929 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.181845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.182444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.182720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.182987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.183673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.185266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.195970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1178/2360] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:38.199946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.210656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c43a2-ffcd-4057-9ff4-34838d63f5a4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.215370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e3a6c-9a8e-4a1c-8b4c-d9e051eca046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.216013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e3a6c-9a8e-4a1c-8b4c-d9e051eca046 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.216289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e3a6c-9a8e-4a1c-8b4c-d9e051eca046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.216559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e3a6c-9a8e-4a1c-8b4c-d9e051eca046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.217193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e3a6c-9a8e-4a1c-8b4c-d9e051eca046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.218829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e3a6c-9a8e-4a1c-8b4c-d9e051eca046 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.231965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1183/2361] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:38.235016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.246396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceab536-fd15-4cd5-af8e-bb9f0273ffbf admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.253552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.254542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.255040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.255397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] PATH_INFO: `/v3/projects/3e6295698830420ab98d5525434cf973/users/dc78b00b9fd14c38aec6fea1d4d54c1f/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.274859 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6295698830420ab98d5525434cf973, user_id=dc78b00b9fd14c38aec6fea1d4d54c1f, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.277725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.292980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1179/2362] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:38 2022] PUT /identity/v3/projects/3e6295698830420ab98d5525434cf973/users/dc78b00b9fd14c38aec6fea1d4d54c1f/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:38.305346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.305616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.305887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.337266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] MFA Rules not processed for user `dc78b00b9fd14c38aec6fea1d4d54c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:38.338837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:38.415287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:38.430229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1184/2363] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:38.442151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.444111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:38.519593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60470e28-d7e0-49a2-8182-4669b0d59df5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.525153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e5bbf-9194-4cf3-8043-ea93b6eeebd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.526036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5bbf-9194-4cf3-8043-ea93b6eeebd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.526476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5bbf-9194-4cf3-8043-ea93b6eeebd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.527499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5bbf-9194-4cf3-8043-ea93b6eeebd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.540275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5bbf-9194-4cf3-8043-ea93b6eeebd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.541928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5bbf-9194-4cf3-8043-ea93b6eeebd6 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.553105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1180/2364] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:38.570859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:38.572480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:38.670112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681edb3d-a44c-44da-a72f-43a6448fb566 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:38.678174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:38.678801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:38.679079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:38.679482 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:38.681339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:38.695418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:38.697109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:38.709584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1185/2365] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:48.548438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:48.704568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=37 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:56.677626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:56.677626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:56.677626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:56.712457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:56.713726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:56.775249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1181/2366] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:56.825791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:56.827842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:56.899006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9990-5e71-48a0-b179-1f519e003f71 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:56.903942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835f40a-2a60-48b2-bfb4-7156107a0e03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:56.904542 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835f40a-2a60-48b2-bfb4-7156107a0e03 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:56.904820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835f40a-2a60-48b2-bfb4-7156107a0e03 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:56.905087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835f40a-2a60-48b2-bfb4-7156107a0e03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:56.913012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835f40a-2a60-48b2-bfb4-7156107a0e03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:56.914578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835f40a-2a60-48b2-bfb4-7156107a0e03 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:56.922952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1186/2367] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:50:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:50:57.551266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:57.655817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a348e-20c1-45a9-88d6-3e99a617d114 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:57.665080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:57.668839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:57.669575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:57.670076 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] PATH_INFO: `/v3/users/dc78b00b9fd14c38aec6fea1d4d54c1f` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:57.672340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc78b00b9fd14c38aec6fea1d4d54c1f)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:57.675969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:57.771967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc78b00b9fd14c38aec6fea1d4d54c1f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:57.783460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc78b00b9fd14c38aec6fea1d4d54c1f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:57.784300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc78b00b9fd14c38aec6fea1d4d54c1f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:57.785092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc78b00b9fd14c38aec6fea1d4d54c1f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:57.793950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc78b00b9fd14c38aec6fea1d4d54c1f'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:57.814676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1182/2368] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:50:57 2022] DELETE /identity/v3/users/dc78b00b9fd14c38aec6fea1d4d54c1f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:57.819460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:57.819650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:57.819936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:57.839333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:57.841390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:57.914581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1187/2369] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:57.928531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:57.929989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:57.964943 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:58.018550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830184b3-a477-4e41-a47b-ec582fec53b4 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:58.022720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821d370-fb4f-45cf-ab43-81e54a4ea2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:58.023992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821d370-fb4f-45cf-ab43-81e54a4ea2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:58.023992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821d370-fb4f-45cf-ab43-81e54a4ea2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:58.024429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821d370-fb4f-45cf-ab43-81e54a4ea2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:58.036059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821d370-fb4f-45cf-ab43-81e54a4ea2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:58.037544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821d370-fb4f-45cf-ab43-81e54a4ea2d2 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:58.050988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1183/2370] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:50:58.401502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:58.403007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:58.448937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f16a0-9c5a-4573-963e-6dbfa97f133a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:58.465442 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:58.466516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:58.467012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:58.468578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] PATH_INFO: `/v3/projects/3e6295698830420ab98d5525434cf973` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:58.470526 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6295698830420ab98d5525434cf973)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:58.478471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:58.544466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6295698830420ab98d5525434cf973', 'request_id': 'req-7771953b-77db-4a62-b7fc-8491b986009b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:58.554514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6295698830420ab98d5525434cf973', 'request_id': 'req-7771953b-77db-4a62-b7fc-8491b986009b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:58.564326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6295698830420ab98d5525434cf973', 'request_id': 'req-7771953b-77db-4a62-b7fc-8491b986009b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:58.577115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] The token cache is being invalidate because project 3e6295698830420ab98d5525434cf973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:50:58.577836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:58.578846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7771953b-77db-4a62-b7fc-8491b986009b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:50:58.580837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1188/2371] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:50:58 2022] DELETE /identity/v3/projects/3e6295698830420ab98d5525434cf973 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:58.585447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:58.585731 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:58.585999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:58.609970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:58.610495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:58.680622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1184/2372] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:58.687236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:58.687793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:58.688090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:58.712759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:58.714273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:58.728076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1189/2373] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:50:58.734112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:58.752812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d1069-f9ff-4e18-9909-5d0f1caae4a3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:58.763948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:58.765420 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:58.766022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:58.766566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:58.768224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:50:58.769124 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:58.772485 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:58.802075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1185/2374] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:50:58.806711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:58.833996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf16546-3173-4a74-8361-36c97cbd2d23 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:58.852487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:58.854789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:58.855490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:58.856167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:58.857558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:58.861870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:58.872973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:58.928704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1190/2375] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:50:58 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:58.932969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:58.952696 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8168f-7a0e-434c-bbbf-a5dc0aaa77a5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:59.053462 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:59.055642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:59.055921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:59.056305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:59.057587 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:59.060011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:59.091465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1186/2376] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:50:58 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:50:59.094743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:59.106594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2da8-9bb1-42d9-b941-b242f90edcf1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:59.112091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:59.112722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:59.113015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:59.113302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:59.114247 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:59.116321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:59.128553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1191/2377] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:59.134347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:59.150423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50f48f-1e7b-4b99-93d2-155079340d85 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:59.155539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c05eab-251e-4349-bebd-31ac98851cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:59.156165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c05eab-251e-4349-bebd-31ac98851cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:59.156444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c05eab-251e-4349-bebd-31ac98851cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:59.156716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c05eab-251e-4349-bebd-31ac98851cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:59.157373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c05eab-251e-4349-bebd-31ac98851cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:59.159273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c05eab-251e-4349-bebd-31ac98851cbf admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:59.173524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1187/2378] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:50:59.179945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:59.202201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03da8-60ac-4017-8c21-61463eb0b18a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:59.209439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:59.210666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:59.211255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:59.211740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] PATH_INFO: `/v3/projects/bd71c6be7e9d4741b129fbc0e6fafad4/users/875e4edde2834c8d9fd40cb219c4d2fd/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:59.233856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd71c6be7e9d4741b129fbc0e6fafad4, user_id=875e4edde2834c8d9fd40cb219c4d2fd, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:59.236632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:59.255946 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1192/2379] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:50:59 2022] PUT /identity/v3/projects/bd71c6be7e9d4741b129fbc0e6fafad4/users/875e4edde2834c8d9fd40cb219c4d2fd/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:50:59.261241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a45f-281d-424e-97d1-a0ebff710b82 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:59.261660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a45f-281d-424e-97d1-a0ebff710b82 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:59.261939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a45f-281d-424e-97d1-a0ebff710b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:59.283812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-3870a45f-281d-424e-97d1-a0ebff710b82 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:50:59.285049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3870a45f-281d-424e-97d1-a0ebff710b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:59.320037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:50:59.359542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1188/2380] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:50:59.369256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:50:59.371120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:50:59.462826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0383a8-20df-46af-a190-063d3a05ad46 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:50:59.470867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:50:59.471943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:50:59.472098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:50:59.472463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:50:59.489769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:50:59.492665 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:50:59.506391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1193/2381] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:51:00.337549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbba5-d96a-4a79-90d1-2a27f9bc836a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:00.338326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbba5-d96a-4a79-90d1-2a27f9bc836a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:00.338326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbba5-d96a-4a79-90d1-2a27f9bc836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:00.382251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4fadbba5-d96a-4a79-90d1-2a27f9bc836a None None] MFA Rules not processed for user `875e4edde2834c8d9fd40cb219c4d2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:00.383863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4fadbba5-d96a-4a79-90d1-2a27f9bc836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:00.464727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1189/2382] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:00.476206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:00.478262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:00.550503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc36e9c-daf1-4ee2-8d64-abd6684401e8 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:00.554324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f2fc-e19f-4dfa-9353-61a5db617513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:00.555124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f2fc-e19f-4dfa-9353-61a5db617513 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:00.555531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f2fc-e19f-4dfa-9353-61a5db617513 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:00.555895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f2fc-e19f-4dfa-9353-61a5db617513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:00.570674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f2fc-e19f-4dfa-9353-61a5db617513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:00.572269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f2fc-e19f-4dfa-9353-61a5db617513 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:00.585233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1194/2383] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:51:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:51:02.098246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9793ffe-2b87-4298-832a-ff88bbab307d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:02.098818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9793ffe-2b87-4298-832a-ff88bbab307d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:02.099096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9793ffe-2b87-4298-832a-ff88bbab307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:02.125669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a9793ffe-2b87-4298-832a-ff88bbab307d None None] MFA Rules not processed for user `875e4edde2834c8d9fd40cb219c4d2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:02.126913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a9793ffe-2b87-4298-832a-ff88bbab307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:02.136670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1190/2384] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:51:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:10.580993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:12.119610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=47 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:12.475599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec909d2-4bc6-4742-8b6f-e5ec58602f3f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:12.477022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec909d2-4bc6-4742-8b6f-e5ec58602f3f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:12.477928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec909d2-4bc6-4742-8b6f-e5ec58602f3f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:12.479842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1195/2385] 10.222.0.55 () {54 vars in 921 bytes} [Sat Sep 24 19:51:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 19:51:12.495140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbcc13-b49c-458a-9fd2-5343e22eaebd None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:12.495445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbcc13-b49c-458a-9fd2-5343e22eaebd None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:12.495730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbcc13-b49c-458a-9fd2-5343e22eaebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:12.516132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0cfbcc13-b49c-458a-9fd2-5343e22eaebd None None] MFA Rules not processed for user `57f68f991883493dbf78c0bcfd827c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:12.517353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0cfbcc13-b49c-458a-9fd2-5343e22eaebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:12.590125 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1191/2386] 10.222.0.55 () {58 vars in 1018 bytes} [Sat Sep 24 19:51:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:12.602321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec909d2-4bc6-4742-8b6f-e5ec58602f3f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:12.604185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fec909d2-4bc6-4742-8b6f-e5ec58602f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:12.664866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec909d2-4bc6-4742-8b6f-e5ec58602f3f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:12.668462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9ce9098e884f4e9119aeccdbafd4ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:12.669073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:12.669601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:12.669888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:12.680333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:12.681755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:12.693701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1196/2387] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:51:15.030504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805647f-7af2-420b-b314-5ef701da8b35 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:15.031402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805647f-7af2-420b-b314-5ef701da8b35 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:15.031882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805647f-7af2-420b-b314-5ef701da8b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:15.053780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9805647f-7af2-420b-b314-5ef701da8b35 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:15.055802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9805647f-7af2-420b-b314-5ef701da8b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:15.133042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1192/2388] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:15.146733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:15.147402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:15.253812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d3466-c55b-489f-8597-db98e73b76b7 service glance] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:15.259447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:15.260060 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:15.260343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:15.260612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:15.269169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:15.270760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:15.279009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1197/2389] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:51:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:51:25.122503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=56 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:25.275698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=58 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:32.670619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:32.671160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:32.671291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:32.693908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:32.695145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:32.702735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1193/2390] 10.222.0.55 () {58 vars in 967 bytes} [Sat Sep 24 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:32.715570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:32.726452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd84f0a-d0f2-4904-bf93-65e1499ac5bb service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:32.732779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:32.733417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:32.733698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:32.733970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:32.742443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:32.744224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:32.753032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1198/2391] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:51:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:51:38.672323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:38.674254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:38.699550 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca140b-a4d6-4919-8556-4868255c7b8f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:38.712986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:38.713795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:38.714163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:38.714572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:38.717244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:38.731890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:38.734276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:38.751781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1194/2392] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:51:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:51:42.749778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:46.299320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:46.300867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:46.313353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceed5c-801a-4e1c-90cb-b3d844cb44f0 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:46.316956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:46.317495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:46.317774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:46.318044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:46.319762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:46.331671 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:46.333106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:46.343412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1199/2393] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:51:48.742404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=9 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:49.403950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:49.405950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:49.421332 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43698dc4-9c68-45e2-ae59-f0ffce4ee202 service nova] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:49.425995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:49.426631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:49.426905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:49.427172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] PATH_INFO: `/v3/users/be2ad8e7eefe447ca1374d49696b1318` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:49.428536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be2ad8e7eefe447ca1374d49696b1318)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:49.430496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:49.488475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be2ad8e7eefe447ca1374d49696b1318'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.496647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be2ad8e7eefe447ca1374d49696b1318'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.497435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be2ad8e7eefe447ca1374d49696b1318'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.498054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be2ad8e7eefe447ca1374d49696b1318'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.505758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-52256366-dace-4831-b1e2-d901fd587d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be2ad8e7eefe447ca1374d49696b1318'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.526451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1195/2394] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:51:49 2022] DELETE /identity/v3/users/be2ad8e7eefe447ca1374d49696b1318 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:51:49.707761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:49.720610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cf905-0b01-4ae0-bdab-7fa381edb3ae service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:49.794943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:49.795707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:49.795988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:49.796259 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] PATH_INFO: `/v3/projects/808b55bc616349cfb31ee041ce69c976` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:49.798054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=808b55bc616349cfb31ee041ce69c976)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:49.800049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:49.850108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808b55bc616349cfb31ee041ce69c976', 'request_id': 'req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.858270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808b55bc616349cfb31ee041ce69c976', 'request_id': 'req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.864356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808b55bc616349cfb31ee041ce69c976', 'request_id': 'req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.876267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] The token cache is being invalidate because project 808b55bc616349cfb31ee041ce69c976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:51:49.876579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.877316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c90c8ed3-ea6b-4133-9d48-9b39173c0cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:51:49.878891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1200/2395] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:51:49 2022] DELETE /identity/v3/projects/808b55bc616349cfb31ee041ce69c976 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:51:49.884187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:49.884674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:49.884776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:49.905822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:49.907032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:49.973340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1196/2396] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:49.983979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:49.984495 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:49.984915 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.007329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:50.008648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:50.017253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1201/2397] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:50.021533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.032709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d629-1474-4ec5-acfb-ad5e41504895 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.038414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.039041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.039313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.039619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.040274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:51:50.040679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.042299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.063541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1197/2398] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:51:50.064606 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.074453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f75285-8e53-4d88-b23a-eea0292c9169 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.079925 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.080568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.081012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.081291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.081973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.084237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.117105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1202/2399] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:51:50 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:51:50.120966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.139801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99083ded-9979-49fa-875d-859a8154eede admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.199160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.199809 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.200092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.200957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.201598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.203572 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.211998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:50.228073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1198/2400] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:51:50 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:51:50.232420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.243036 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3f67-3840-4b42-8741-4d68cddcd028 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.248948 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.249554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.249835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.250107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.250757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.252660 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.263945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1203/2401] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:51:50.268096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.278515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d537ba-4b38-4aa2-baab-902ef7209d57 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.283710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130c2f8-e22f-4fa5-b055-f3e874b3404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.284371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130c2f8-e22f-4fa5-b055-f3e874b3404a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.284652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130c2f8-e22f-4fa5-b055-f3e874b3404a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.284927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130c2f8-e22f-4fa5-b055-f3e874b3404a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.285571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130c2f8-e22f-4fa5-b055-f3e874b3404a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.287196 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130c2f8-e22f-4fa5-b055-f3e874b3404a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.298875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1199/2402] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:51:50.303102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.314093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285524b-4cfe-46b1-aba0-7fa59d1937dd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.319156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.319839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.320122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.320422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] PATH_INFO: `/v3/projects/6c912f342b8344c1a0849ebde207f356/users/47bd2fa354b74fbf8d518da924ca1bfa/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.336094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c912f342b8344c1a0849ebde207f356, user_id=47bd2fa354b74fbf8d518da924ca1bfa, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.338749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.352263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1204/2403] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:51:50 2022] PUT /identity/v3/projects/6c912f342b8344c1a0849ebde207f356/users/47bd2fa354b74fbf8d518da924ca1bfa/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:51:50.362782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.363075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.363331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.392159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] MFA Rules not processed for user `47bd2fa354b74fbf8d518da924ca1bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:51:50.393370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:50.458884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1200/2404] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:51:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:51:50.470315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:51:50.472216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:51:50.488091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:51:50.533428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94b95c-e030-468b-82c4-fbd03e3fd325 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:51:50.537216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861e940-60e6-43c6-bd2c-c4cee346dc51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:51:50.537825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e940-60e6-43c6-bd2c-c4cee346dc51 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:51:50.538109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e940-60e6-43c6-bd2c-c4cee346dc51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:51:50.538382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e940-60e6-43c6-bd2c-c4cee346dc51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:51:50.547690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e940-60e6-43c6-bd2c-c4cee346dc51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:51:50.549105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e940-60e6-43c6-bd2c-c4cee346dc51 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:51:50.557868 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1205/2405] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:51:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:00.456202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:00.553889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=20 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:01.316455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:01.394655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672b6f6-7b34-4fdd-a2a7-ae61a5f28a64 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:01.402563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:01.403860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:01.404146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:01.404424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] PATH_INFO: `/v3/users/47bd2fa354b74fbf8d518da924ca1bfa` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:01.405678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bd2fa354b74fbf8d518da924ca1bfa)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:01.407722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:01.479863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bd2fa354b74fbf8d518da924ca1bfa'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:01.490867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bd2fa354b74fbf8d518da924ca1bfa'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:01.491900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bd2fa354b74fbf8d518da924ca1bfa'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:01.492718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bd2fa354b74fbf8d518da924ca1bfa'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:01.501011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bd2fa354b74fbf8d518da924ca1bfa'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:01.520360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1201/2406] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:01 2022] DELETE /identity/v3/users/47bd2fa354b74fbf8d518da924ca1bfa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:01.525676 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:01.525958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:01.526229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:01.548080 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:01.549321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:01.626395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1206/2407] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:01.636409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:01.637947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:01.698942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9471c-7905-42d1-a25c-e558a99c84a0 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:01.703932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea7605-3650-4248-8696-b4ec1c84cbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:01.704557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea7605-3650-4248-8696-b4ec1c84cbdc service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:01.704825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea7605-3650-4248-8696-b4ec1c84cbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:01.705147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea7605-3650-4248-8696-b4ec1c84cbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:01.720511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea7605-3650-4248-8696-b4ec1c84cbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:01.723112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea7605-3650-4248-8696-b4ec1c84cbdc service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:01.738575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1202/2408] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:02.167395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.170074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:02.200489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb63eb-112a-4b5c-a150-efdc36361765 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.209759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.210721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.211213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.211847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] PATH_INFO: `/v3/projects/6c912f342b8344c1a0849ebde207f356` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.213308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c912f342b8344c1a0849ebde207f356)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.216409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.354638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c912f342b8344c1a0849ebde207f356', 'request_id': 'req-75b3488a-55ce-4213-9b78-86e9840cf122'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:02.372895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c912f342b8344c1a0849ebde207f356', 'request_id': 'req-75b3488a-55ce-4213-9b78-86e9840cf122'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:02.388620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c912f342b8344c1a0849ebde207f356', 'request_id': 'req-75b3488a-55ce-4213-9b78-86e9840cf122'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:02.408819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] The token cache is being invalidate because project 6c912f342b8344c1a0849ebde207f356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:02.409115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:02.409853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-75b3488a-55ce-4213-9b78-86e9840cf122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:02.411702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1207/2409] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:02 2022] DELETE /identity/v3/projects/6c912f342b8344c1a0849ebde207f356 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:02.416622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.417056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.417424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.440682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:02.442074 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:02.464159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:02.521287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1203/2410] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:02.529258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.529642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.529962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.555149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:02.556428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:02.565399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1208/2411] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:02.569941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.581389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f81397-bca8-47ce-b9cd-fa1db312e9af None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.589075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.590233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.590777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.591289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.592109 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:02.592521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.594324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.610639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1204/2412] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:02.615534 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.625937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67053b-393c-4182-9d1b-4bd4e1dc359c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.630863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.631569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.631777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.632064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.632709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.634642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.665991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1209/2413] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:02 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:02.670885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.682218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1fc45-bb4b-41a0-b79a-d19c0fe3c351 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.741539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.742152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.742429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.742698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.743346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.745313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.769343 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1205/2414] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:02 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:02.773671 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.790350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd736a-a883-4716-b23b-30a5970de643 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.795618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.796351 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.796638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.796908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.797558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.803098 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.811766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1210/2415] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:02.815891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.833416 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af76dc-ab27-4acc-ad9d-74a5359ff988 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.838157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c71fa-fbb4-4924-b12c-36c464cef8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.838767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c71fa-fbb4-4924-b12c-36c464cef8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.839044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c71fa-fbb4-4924-b12c-36c464cef8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.839340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c71fa-fbb4-4924-b12c-36c464cef8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.840019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c71fa-fbb4-4924-b12c-36c464cef8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.842395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c71fa-fbb4-4924-b12c-36c464cef8f9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.854364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1206/2416] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:02.859467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:02.870433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70906fc-e771-410f-8d00-dafbfb1daf06 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:02.875818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:02.876487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.876765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.877038 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] PATH_INFO: `/v3/projects/d09ffc25d4744e398005e441e0d4b057/users/a8e82816d8ad4fa88bcfb5a59dfd72a1/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.892536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09ffc25d4744e398005e441e0d4b057, user_id=a8e82816d8ad4fa88bcfb5a59dfd72a1, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:02.895204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:02.911097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1211/2417] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:02 2022] PUT /identity/v3/projects/d09ffc25d4744e398005e441e0d4b057/users/a8e82816d8ad4fa88bcfb5a59dfd72a1/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:02.922951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:02.923218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:02.923523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:02.956086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] MFA Rules not processed for user `a8e82816d8ad4fa88bcfb5a59dfd72a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:02.957425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:03.034894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1207/2418] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:03.039530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.064054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:03.118431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b8ee1-e9b6-401e-ad92-d2223677e9bb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.124079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.124821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.125174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.125533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.126245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.128291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.158886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1212/2419] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:03 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:03.163415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.175844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b5c5-cb04-44b3-99d2-4fed0ba53cb3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.189814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:03.240057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.240705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.240989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.241265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.241905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.243889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.269210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1208/2420] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:03 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:03.273346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.285535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188cc1-fbbf-436b-b963-b68867798bf7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.291712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.292474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.292828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.293185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.294900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.296722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.308470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1213/2421] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:03.313677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.324832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5c02-28fe-4f11-84aa-5cb677c418d6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.330783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.331495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.331883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.332051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] PATH_INFO: `/v3/projects/dbefdeeb5d5e44a5a5f542353273097b/users/5ead9b2a489f455b83382a94087ca647/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.348340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbefdeeb5d5e44a5a5f542353273097b, user_id=5ead9b2a489f455b83382a94087ca647, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.350690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.367736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1209/2422] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:03 2022] PUT /identity/v3/projects/dbefdeeb5d5e44a5a5f542353273097b/users/5ead9b2a489f455b83382a94087ca647/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:03.372557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.382974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e734e-b868-451b-9e56-6eba6dc93abe admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.447650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1e9d8-3452-4ff9-b732-b57927b38935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.448176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e9d8-3452-4ff9-b732-b57927b38935 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.448456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e9d8-3452-4ff9-b732-b57927b38935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.448728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e9d8-3452-4ff9-b732-b57927b38935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.449369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1e9d8-3452-4ff9-b732-b57927b38935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.451157 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1e9d8-3452-4ff9-b732-b57927b38935 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.461371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1214/2423] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:03.466457 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.478137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570906c-30aa-4855-8027-9d41a60e1064 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.483675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.484351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.484633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.484910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] PATH_INFO: `/v3/projects/dbefdeeb5d5e44a5a5f542353273097b/users/5ead9b2a489f455b83382a94087ca647/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.486867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbefdeeb5d5e44a5a5f542353273097b, user_id=5ead9b2a489f455b83382a94087ca647, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.489613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.501088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1210/2424] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:03 2022] PUT /identity/v3/projects/dbefdeeb5d5e44a5a5f542353273097b/users/5ead9b2a489f455b83382a94087ca647/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:03.513209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.513493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.513958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.542260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] MFA Rules not processed for user `5ead9b2a489f455b83382a94087ca647`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:03.544618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:03.623489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1215/2425] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:03.634863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.637044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:03.701555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6be8-ff0b-46f8-ae81-c5562521612e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.705238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dbb04-b524-44e2-8023-65c46e57f1a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.705858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dbb04-b524-44e2-8023-65c46e57f1a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.706133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dbb04-b524-44e2-8023-65c46e57f1a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.706408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dbb04-b524-44e2-8023-65c46e57f1a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.717734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dbb04-b524-44e2-8023-65c46e57f1a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.719193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dbb04-b524-44e2-8023-65c46e57f1a5 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.729700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1211/2426] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:03.815851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:03.826175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c418b-948b-491d-988e-8bcde29e4e7e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:03.830028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:03.830740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:03.831082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:03.831419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:03.846334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:03.906374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:03.907865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:03.922821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1216/2427] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:13.725099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:13.916700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=21 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:16.863369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecc130-8642-4bcf-98e9-0ffe479aa153 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:16.863958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecc130-8642-4bcf-98e9-0ffe479aa153 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:16.863958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecc130-8642-4bcf-98e9-0ffe479aa153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:16.886101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-8eecc130-8642-4bcf-98e9-0ffe479aa153 None None] MFA Rules not processed for user `875e4edde2834c8d9fd40cb219c4d2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:16.888357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8eecc130-8642-4bcf-98e9-0ffe479aa153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:16.948745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1212/2428] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:18.085026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:18.157268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fa137-d945-4502-a118-1134d30bc30b service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:18.161864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:18.162498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:18.162773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:18.163040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] PATH_INFO: `/v3/users/a8e82816d8ad4fa88bcfb5a59dfd72a1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:18.164188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e82816d8ad4fa88bcfb5a59dfd72a1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:18.166133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:18.223190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e82816d8ad4fa88bcfb5a59dfd72a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.236136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e82816d8ad4fa88bcfb5a59dfd72a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.243846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e82816d8ad4fa88bcfb5a59dfd72a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.244455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e82816d8ad4fa88bcfb5a59dfd72a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.252527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e82816d8ad4fa88bcfb5a59dfd72a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.259594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1217/2429] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:18 2022] DELETE /identity/v3/users/a8e82816d8ad4fa88bcfb5a59dfd72a1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:18.264253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:18.264780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:18.264878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:18.283895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:18.285138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:18.359913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1213/2430] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:18.371601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:18.380556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:18.425276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:18.427101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:18.465787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa8a31-e9cf-4e2a-9174-a7738da80e44 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:18.473250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:18.474135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:18.474594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:18.475017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:18.486395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:18.487957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:18.498869 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1218/2431] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:18.500313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21656661-e266-41f8-9ab3-6fc6e5b44985 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:18.504011 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:18.504627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:18.504900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:18.505193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:18.506951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:18.519464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:18.520942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:18.532326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1214/2432] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:18.819116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:18.820610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:18.834498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a1d6-2ae9-40f5-8685-8d0d58e758f2 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:18.839548 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:18.840724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:18.841042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:18.841316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] PATH_INFO: `/v3/projects/d09ffc25d4744e398005e441e0d4b057` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:18.842500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09ffc25d4744e398005e441e0d4b057)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:18.844525 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:18.865119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:18.943232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09ffc25d4744e398005e441e0d4b057', 'request_id': 'req-9d11a545-7d73-40aa-939c-3d77caa7cf2f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.959066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09ffc25d4744e398005e441e0d4b057', 'request_id': 'req-9d11a545-7d73-40aa-939c-3d77caa7cf2f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.970712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09ffc25d4744e398005e441e0d4b057', 'request_id': 'req-9d11a545-7d73-40aa-939c-3d77caa7cf2f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.984939 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] The token cache is being invalidate because project d09ffc25d4744e398005e441e0d4b057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:18.985381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.986134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:18.987908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1219/2433] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:18 2022] DELETE /identity/v3/projects/d09ffc25d4744e398005e441e0d4b057 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:18.993500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:19.008487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e05f08-1f5c-4fe3-a2a2-051ca6157820 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:19.073580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:19.098073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:19.100123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:19.100725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:19.101282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] PATH_INFO: `/v3/users/5ead9b2a489f455b83382a94087ca647` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:19.102662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ead9b2a489f455b83382a94087ca647)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:19.105645 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:19.171978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ead9b2a489f455b83382a94087ca647'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:19.182781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ead9b2a489f455b83382a94087ca647'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:19.183783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ead9b2a489f455b83382a94087ca647'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:19.185301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ead9b2a489f455b83382a94087ca647'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:19.196704 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c9becf35-677c-4049-9f17-6ca39de90bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ead9b2a489f455b83382a94087ca647'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:19.239372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1215/2434] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:18 2022] DELETE /identity/v3/users/5ead9b2a489f455b83382a94087ca647 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:19.888448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:19.890122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:19.907455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11a545-7d73-40aa-939c-3d77caa7cf2f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.000458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.001042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.001322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.001599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] PATH_INFO: `/v3/projects/dbefdeeb5d5e44a5a5f542353273097b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.002761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbefdeeb5d5e44a5a5f542353273097b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.005733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.074824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbefdeeb5d5e44a5a5f542353273097b', 'request_id': 'req-63ddc847-ed0f-4567-9aec-0da9d89940ca'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:20.084190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbefdeeb5d5e44a5a5f542353273097b', 'request_id': 'req-63ddc847-ed0f-4567-9aec-0da9d89940ca'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:20.091973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbefdeeb5d5e44a5a5f542353273097b', 'request_id': 'req-63ddc847-ed0f-4567-9aec-0da9d89940ca'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:20.107331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] The token cache is being invalidate because project dbefdeeb5d5e44a5a5f542353273097b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:20.107642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:20.108424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-63ddc847-ed0f-4567-9aec-0da9d89940ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:20.110234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1220/2435] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:19 2022] DELETE /identity/v3/projects/dbefdeeb5d5e44a5a5f542353273097b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:20.115300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.115916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.115916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.139527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:20.140954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:20.220154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1216/2436] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:20.228454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.228782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.229044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.252742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:20.254068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:20.269083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1221/2437] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:20.273533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.285227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d2d4e-64dd-451c-aedb-f9491a9637f0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.291650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.292584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.292888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.293947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.295301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:20.295624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.297499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.314050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1217/2438] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:20.318549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.332192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d086a-34ed-4b8b-aae0-ea6ed61037f3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.339735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.341398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.341688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.342356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.342635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.345349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.380643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1222/2439] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:20 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:20.385714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.398762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09d100-167f-4213-a253-2ac7f9d77482 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.467203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:20.476996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.478531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.478531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.478743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.479799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.482502 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.554286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1218/2440] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:20 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:20.560950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.577379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ebba6-e8de-484c-86e7-99bf1da8cfe9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.587668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.588694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.589206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.591669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.593974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.595549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.597407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:20.617702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1223/2441] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:20.623047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.635549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231da6b-3760-4c19-b25a-4552a941230a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.641896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8eee31-9b19-4032-9b2d-e3fe26b854e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.642510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8eee31-9b19-4032-9b2d-e3fe26b854e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.642792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8eee31-9b19-4032-9b2d-e3fe26b854e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.643065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8eee31-9b19-4032-9b2d-e3fe26b854e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.643846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8eee31-9b19-4032-9b2d-e3fe26b854e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.646067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8eee31-9b19-4032-9b2d-e3fe26b854e5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.659718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1219/2442] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:20.663885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.678342 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e48e-218f-4451-8d2e-f4de6be25044 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.685104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.685824 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.686197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.686563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] PATH_INFO: `/v3/projects/02e63317e10d45dc8b20dfb485008ded/users/d21110cc45084aec964768aab2990a2a/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.704822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e63317e10d45dc8b20dfb485008ded, user_id=d21110cc45084aec964768aab2990a2a, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.707822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:20.737493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1224/2443] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:20 2022] PUT /identity/v3/projects/02e63317e10d45dc8b20dfb485008ded/users/d21110cc45084aec964768aab2990a2a/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:20.749556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.749710 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.749982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.779982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] MFA Rules not processed for user `d21110cc45084aec964768aab2990a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:20.781146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:20.873445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1220/2444] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:20.878333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:20.963292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7860da5-15f1-4f68-9df9-fbf815b195bc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:20.970854 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:20.971511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:20.971794 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:20.972067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:20.973317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:20.975923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.011565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1225/2445] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:20 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:21.016210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:21.028869 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553c7b4-ddb8-44cd-ba74-b0db75788a40 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:21.119323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:21.120209 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.120709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.121629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.122742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:21.125603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.160761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1221/2446] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:21 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:21.162510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:21.178645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed791c-9de7-4a32-b13b-cd3f2b20d10b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:21.185569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:21.186191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.186468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.186739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.187388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:21.189054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.200727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1226/2447] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:21.205541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:21.217463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db39b5-be07-4c10-bc4a-d127fe6b2a00 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:21.223132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:21.223913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.224215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.224636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] PATH_INFO: `/v3/projects/f9d2caadcf9a4c829b82ab672857e2ab/users/c880f214003c4fb6b8c3da08aca85884/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.242053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d2caadcf9a4c829b82ab672857e2ab, user_id=c880f214003c4fb6b8c3da08aca85884, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:21.245072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.262283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1222/2448] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:21 2022] PUT /identity/v3/projects/f9d2caadcf9a4c829b82ab672857e2ab/users/c880f214003c4fb6b8c3da08aca85884/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:21.269007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:21.281934 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73862d-a4d7-4e7e-99f5-82019a08cd70 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:21.348698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4c668-c620-40be-9678-f9334959ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:21.349464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c668-c620-40be-9678-f9334959ca7b admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.349808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c668-c620-40be-9678-f9334959ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.350154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c668-c620-40be-9678-f9334959ca7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.350839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4c668-c620-40be-9678-f9334959ca7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:21.352569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4c668-c620-40be-9678-f9334959ca7b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.368505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1227/2449] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:21.373501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:21.384200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ad2d8-9aed-45ab-b4b0-94243df1bbf6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:21.389244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:21.389976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.390319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.390675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] PATH_INFO: `/v3/projects/f9d2caadcf9a4c829b82ab672857e2ab/users/c880f214003c4fb6b8c3da08aca85884/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.392689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d2caadcf9a4c829b82ab672857e2ab, user_id=c880f214003c4fb6b8c3da08aca85884, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:21.395470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.406450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1223/2450] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:21 2022] PUT /identity/v3/projects/f9d2caadcf9a4c829b82ab672857e2ab/users/c880f214003c4fb6b8c3da08aca85884/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:21.416762 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.417051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.417324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.450467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] MFA Rules not processed for user `c880f214003c4fb6b8c3da08aca85884`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:21.451956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:21.483581 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:21.523347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1228/2451] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:21.536556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:21.538236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:21.564214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:21.661569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594926d3-7d6b-4bfa-9192-5e33973637af admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:21.665778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645900-7b08-4ddd-b588-46ee09bb6ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:21.666403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645900-7b08-4ddd-b588-46ee09bb6ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:21.666677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645900-7b08-4ddd-b588-46ee09bb6ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:21.666942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645900-7b08-4ddd-b588-46ee09bb6ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:21.687775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645900-7b08-4ddd-b588-46ee09bb6ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:21.690439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645900-7b08-4ddd-b588-46ee09bb6ad9 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:21.705293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1224/2452] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:22.251926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:22.335096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9772fc-8ab3-46ea-981e-f55f0d611380 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:22.340763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:22.341958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:22.342348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:22.342756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] PATH_INFO: `/v3/users/d21110cc45084aec964768aab2990a2a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:22.344257 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21110cc45084aec964768aab2990a2a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:22.346668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:22.413650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21110cc45084aec964768aab2990a2a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:22.430980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21110cc45084aec964768aab2990a2a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:22.441956 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21110cc45084aec964768aab2990a2a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:22.442741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21110cc45084aec964768aab2990a2a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:22.469215 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21110cc45084aec964768aab2990a2a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:22.489054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1229/2453] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:22 2022] DELETE /identity/v3/users/d21110cc45084aec964768aab2990a2a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:22.494191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:22.494672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:22.497281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:22.545913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:22.545913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:22.647065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1225/2454] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:22.659373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:22.661105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:22.740599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:22.750691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66202b16-f03e-4910-8fa7-d8ea742dafb0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:22.755346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:22.756066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:22.756412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:22.757142 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:22.768610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:22.770619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:22.785405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1230/2455] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:23.206421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.208132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:23.222962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e9f26-591f-43b1-8b5f-bc4de4c7211d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.232182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:23.232874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.233163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.233453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] PATH_INFO: `/v3/users/444719e34bf2457b9a58e182ef7fbd79` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.236185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=444719e34bf2457b9a58e182ef7fbd79)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:23.238512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:23.284624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.286139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:23.290448 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444719e34bf2457b9a58e182ef7fbd79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.301610 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444719e34bf2457b9a58e182ef7fbd79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.302378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444719e34bf2457b9a58e182ef7fbd79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.303210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444719e34bf2457b9a58e182ef7fbd79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.303339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e1440-dd60-4a1c-b8b3-ecb4d30472c8 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.308216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:23.309017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.309354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.309709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] PATH_INFO: `/v3/projects/02e63317e10d45dc8b20dfb485008ded` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.310974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e63317e10d45dc8b20dfb485008ded)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:23.311990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444719e34bf2457b9a58e182ef7fbd79'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.313160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:23.331720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1226/2456] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:23 2022] DELETE /identity/v3/users/444719e34bf2457b9a58e182ef7fbd79 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:23.367949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e63317e10d45dc8b20dfb485008ded', 'request_id': 'req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.376093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e63317e10d45dc8b20dfb485008ded', 'request_id': 'req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.382815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e63317e10d45dc8b20dfb485008ded', 'request_id': 'req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.394959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] The token cache is being invalidate because project 02e63317e10d45dc8b20dfb485008ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:23.395245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.396099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.397856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1231/2457] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:23 2022] DELETE /identity/v3/projects/02e63317e10d45dc8b20dfb485008ded => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:23.401939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.403952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:23.440912 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:23.476491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f852-a180-4a4a-aca5-0841edbee7ff admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.524265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.525760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:23.555611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:23.556277 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.556517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.556963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] PATH_INFO: `/v3/users/c880f214003c4fb6b8c3da08aca85884` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.558063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c880f214003c4fb6b8c3da08aca85884)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:23.563255 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:23.612223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb702e-95a3-4c6e-b833-5cf03e656d8e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.619012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:23.619680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.620021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.620472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] PATH_INFO: `/v3/projects/977038321e5a41e8ab137fede4d118d7` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.622027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=977038321e5a41e8ab137fede4d118d7)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:23.624701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:23.628918 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c880f214003c4fb6b8c3da08aca85884'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.639322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c880f214003c4fb6b8c3da08aca85884'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.640071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c880f214003c4fb6b8c3da08aca85884'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.640714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c880f214003c4fb6b8c3da08aca85884'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.648135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6ca408a2-0836-433c-8715-453e0278f83c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c880f214003c4fb6b8c3da08aca85884'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.668199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1227/2458] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:23 2022] DELETE /identity/v3/users/c880f214003c4fb6b8c3da08aca85884 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:23.677293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977038321e5a41e8ab137fede4d118d7', 'request_id': 'req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.685169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977038321e5a41e8ab137fede4d118d7', 'request_id': 'req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.691655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977038321e5a41e8ab137fede4d118d7', 'request_id': 'req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.704012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] The token cache is being invalidate because project 977038321e5a41e8ab137fede4d118d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:23.704383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.705039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-12db1bdf-3608-4f34-b0c1-54fe34c5dd2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:23.706530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1232/2459] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:23 2022] DELETE /identity/v3/projects/977038321e5a41e8ab137fede4d118d7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:23.712233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001e82-5375-4574-a723-81669978ed8d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.712518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001e82-5375-4574-a723-81669978ed8d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.712788 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001e82-5375-4574-a723-81669978ed8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.735415 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-09001e82-5375-4574-a723-81669978ed8d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:23.736857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-09001e82-5375-4574-a723-81669978ed8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:23.797257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:23.816774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1228/2460] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:23.818806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.819357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.820000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.830733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:23.851087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:23.852353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:23.861942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1233/2461] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:23.866345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09001e82-5375-4574-a723-81669978ed8d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.877162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09001e82-5375-4574-a723-81669978ed8d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.882774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:23.883505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.883854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.884198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.884984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:23.885346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:23.887600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:23.904934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1229/2462] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:23.906714 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.917499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853e1ac-b61a-4703-b812-fb86b66b806f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.923332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:23.924074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:23.924387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:23.924661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:23.925296 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:23.927204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:23.963209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1234/2463] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:23 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:23.968179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.978947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55788-69c6-4a63-8d71-330398e62be2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:23.989564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:23.991463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.064120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.064747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.065026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.065303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.070852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.072928 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.082419 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c6740-9478-4258-9eb2-c52abf854fbc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.089338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.090616 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.091202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.091590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] PATH_INFO: `/v3/projects/f9d2caadcf9a4c829b82ab672857e2ab` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.092907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d2caadcf9a4c829b82ab672857e2ab)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.094848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.099494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1230/2464] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:23 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:24.103302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.115466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f2231-5430-4ce1-8807-6ad967cdb1d4 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.120964 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.121772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.122090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.122372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.123858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.125578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.157878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1231/2465] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:24.162134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.187143 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb77a73-77f3-42be-b23f-a902aecd8323 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.198119 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d2caadcf9a4c829b82ab672857e2ab', 'request_id': 'req-a52db944-4281-47e1-b1cf-1da03c5d2f42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:24.216053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d2caadcf9a4c829b82ab672857e2ab', 'request_id': 'req-a52db944-4281-47e1-b1cf-1da03c5d2f42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:24.224450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d2caadcf9a4c829b82ab672857e2ab', 'request_id': 'req-a52db944-4281-47e1-b1cf-1da03c5d2f42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:24.241512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] The token cache is being invalidate because project f9d2caadcf9a4c829b82ab672857e2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:24.241807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:24.242523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a52db944-4281-47e1-b1cf-1da03c5d2f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:24.244324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1235/2466] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:23 2022] DELETE /identity/v3/projects/f9d2caadcf9a4c829b82ab672857e2ab => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:24.250271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7aee6-6c93-4cd1-b11a-04618354107f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.250559 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7aee6-6c93-4cd1-b11a-04618354107f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.250836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7aee6-6c93-4cd1-b11a-04618354107f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.276748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bab7aee6-6c93-4cd1-b11a-04618354107f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:24.278655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bab7aee6-6c93-4cd1-b11a-04618354107f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.300193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.301539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.301821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.302089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.302751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.304623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.325796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1232/2467] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:24.326732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.336906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.351475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa066556-1a54-42b4-a807-a587b95fe4c2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.356594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.357428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.359473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.359643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1236/2468] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:24.360081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] PATH_INFO: `/v3/projects/85d897789ca8446eaa73a4c4f6e7d677/users/e871e64713dc46b7b09d1d7832dd2e83/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.365807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d602ca9-b3be-4977-976a-61242929480d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.366438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d602ca9-b3be-4977-976a-61242929480d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.367398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d602ca9-b3be-4977-976a-61242929480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.378839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:24.385214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d897789ca8446eaa73a4c4f6e7d677, user_id=e871e64713dc46b7b09d1d7832dd2e83, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.389283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.396409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8d602ca9-b3be-4977-976a-61242929480d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:24.399379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8d602ca9-b3be-4977-976a-61242929480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.409561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1233/2469] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:24 2022] PUT /identity/v3/projects/85d897789ca8446eaa73a4c4f6e7d677/users/e871e64713dc46b7b09d1d7832dd2e83/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:24.412270 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1237/2470] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:52:24.416574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.420412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.420853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.421245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.460836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] MFA Rules not processed for user `e871e64713dc46b7b09d1d7832dd2e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:24.461609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:24.462051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.505479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d64b3-a752-45fe-a1a9-c8672158c748 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.510373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.511107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.511527 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.512014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.512759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:24.513159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.514853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.527055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1234/2471] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:52:24.532630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.535137 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1238/2472] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:52:24.546426 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.547293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9583-6db8-4583-96fd-6d74d3357f32 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.548582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.554148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.554899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.555275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.555679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.556426 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.559848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.598574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1235/2473] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/projects => generated 355 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:24.599715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.641014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774eda37-8f45-4889-9b00-ad6172c9d394 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.684360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776e1eb-bcdd-42b4-b680-3cbb14329a72 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.689670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.690285 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.690558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.691101 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.691600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.693668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.717029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1236/2474] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/users => generated 278 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:24.719790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e54c40-ff12-4c6e-91df-ebf8d8bf6363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.720507 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54c40-ff12-4c6e-91df-ebf8d8bf6363 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.720919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54c40-ff12-4c6e-91df-ebf8d8bf6363 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.721033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.721362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54c40-ff12-4c6e-91df-ebf8d8bf6363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.733460 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b617c-f825-4e02-bd5e-87c96f10d21c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.742608 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.743219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.743528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.744023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:24.744170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.744719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.747403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.761758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1237/2475] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:24.766013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.776546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68430b-0d39-4d44-8fc1-a04991c2c00f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.781359 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.781981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.782263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.782662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.783520 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.784836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.786356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e54c40-ff12-4c6e-91df-ebf8d8bf6363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.788392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e54c40-ff12-4c6e-91df-ebf8d8bf6363 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.797027 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1238/2476] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:24.801188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:24.802236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1239/2477] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:24.812418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c782d-d08f-469d-bd50-d38ff4eb5ddb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:24.818609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:24.819367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.819803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.820588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] PATH_INFO: `/v3/projects/3dc2cb99fd7d45479529ba277e5d1321/users/e2484143c56d4629a5466f5504d2e869/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.836519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc2cb99fd7d45479529ba277e5d1321, user_id=e2484143c56d4629a5466f5504d2e869, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:24.839397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:24.853294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1239/2478] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:24 2022] PUT /identity/v3/projects/3dc2cb99fd7d45479529ba277e5d1321/users/e2484143c56d4629a5466f5504d2e869/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:24.866284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:24.866776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:24.867460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:24.900872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] MFA Rules not processed for user `e2484143c56d4629a5466f5504d2e869`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:24.901661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:24.993591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1240/2479] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:24.999035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.064494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:25.081582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8823c5-179b-4bb1-9063-19b8519c16f8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.088383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.089324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.089611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.090058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.091193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.093646 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.130523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1240/2480] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:24 2022] POST /identity/v3/projects => generated 355 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:25.135106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.148466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccfdf3-9e46-417d-8073-8c7cd38021ae None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.244843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.245600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.246899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.247356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.248359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.251027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.287736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1241/2481] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:25 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:25.289202 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.302085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49adb9c-e1f4-4434-962c-62c575836072 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.309682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.312174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.312174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.312174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.312632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.314096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.326623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1241/2482] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:25.331344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.341822 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa63f0-47ba-433a-9099-28ab5691cede admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.347147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.347849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.348126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.348555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] PATH_INFO: `/v3/projects/6a8a897925a6482781e53bfdbfb2b543/users/50ef4943b58f4cbb8145756a3420500a/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.365025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8a897925a6482781e53bfdbfb2b543, user_id=50ef4943b58f4cbb8145756a3420500a, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.367748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.381254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1242/2483] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:25 2022] PUT /identity/v3/projects/6a8a897925a6482781e53bfdbfb2b543/users/50ef4943b58f4cbb8145756a3420500a/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:25.385287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.402595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c38a32-12f7-4e9b-a910-bd74c8507365 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.509915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10127d8b-3626-48db-a267-cd791b79ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.510944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10127d8b-3626-48db-a267-cd791b79ee8b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.511254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10127d8b-3626-48db-a267-cd791b79ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.511254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10127d8b-3626-48db-a267-cd791b79ee8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.513254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10127d8b-3626-48db-a267-cd791b79ee8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.515598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10127d8b-3626-48db-a267-cd791b79ee8b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.544636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1242/2484] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:25.548588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.566569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4e6c-206c-4a46-b545-f6807603aea8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.573552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.575564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.576224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.576789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] PATH_INFO: `/v3/projects/6a8a897925a6482781e53bfdbfb2b543/users/50ef4943b58f4cbb8145756a3420500a/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.579689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8a897925a6482781e53bfdbfb2b543, user_id=50ef4943b58f4cbb8145756a3420500a, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.583441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.617460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1243/2485] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:25 2022] PUT /identity/v3/projects/6a8a897925a6482781e53bfdbfb2b543/users/50ef4943b58f4cbb8145756a3420500a/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:25.628516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.629025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.629119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.675968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] MFA Rules not processed for user `50ef4943b58f4cbb8145756a3420500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:25.677391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:25.826184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1243/2486] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:25.837590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:25.839978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:25.846385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:25.909608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af285e31-74c0-4dd7-84e1-65934d641b28 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:25.914380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:25.915001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:25.915280 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:25.915586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:25.926053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:25.927410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:25.938078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1244/2487] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:26.024449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:26.035862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ec4ac-030c-4c60-b5f2-1eb40a3e0988 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:26.039817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa005b1-6dee-4c3d-88aa-db4a768a5d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:26.040456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa005b1-6dee-4c3d-88aa-db4a768a5d78 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:26.040744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa005b1-6dee-4c3d-88aa-db4a768a5d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:26.041009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa005b1-6dee-4c3d-88aa-db4a768a5d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:26.101679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:26.117651 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa005b1-6dee-4c3d-88aa-db4a768a5d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:26.119159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa005b1-6dee-4c3d-88aa-db4a768a5d78 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:26.129927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1244/2488] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:27.664708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:27.737590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dba00-cced-4f07-ae9b-2a0ab5d0b7a3 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:27.742272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:27.742910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:27.743191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:27.743488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] PATH_INFO: `/v3/users/e871e64713dc46b7b09d1d7832dd2e83` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:27.744626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e871e64713dc46b7b09d1d7832dd2e83)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:27.747066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:27.822836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e871e64713dc46b7b09d1d7832dd2e83'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:27.845470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e871e64713dc46b7b09d1d7832dd2e83'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:27.854718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e871e64713dc46b7b09d1d7832dd2e83'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:27.856009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e871e64713dc46b7b09d1d7832dd2e83'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:27.872834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e871e64713dc46b7b09d1d7832dd2e83'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:27.881352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1245/2489] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:27 2022] DELETE /identity/v3/users/e871e64713dc46b7b09d1d7832dd2e83 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:27.886115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:27.887184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:27.887184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:27.909507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:27.910852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:27.989351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1245/2490] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:28.002565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:28.004030 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:28.041177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:28.086708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1fef6-380a-4507-85f6-43ec8cc26fae admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:28.090754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3daa68-c4e4-43c7-8199-aa04b6bf96fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:28.091372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3daa68-c4e4-43c7-8199-aa04b6bf96fc service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:28.091733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3daa68-c4e4-43c7-8199-aa04b6bf96fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:28.092004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3daa68-c4e4-43c7-8199-aa04b6bf96fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:28.102573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3daa68-c4e4-43c7-8199-aa04b6bf96fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:28.104088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3daa68-c4e4-43c7-8199-aa04b6bf96fc service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:28.125654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1246/2491] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:28.485301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:28.487649 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:28.503533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32058cc4-7dc3-455b-a6fa-439b8796ce16 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:28.509707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:28.510474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:28.510753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:28.511135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] PATH_INFO: `/v3/projects/85d897789ca8446eaa73a4c4f6e7d677` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:28.512677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d897789ca8446eaa73a4c4f6e7d677)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:28.514714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:28.572953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d897789ca8446eaa73a4c4f6e7d677', 'request_id': 'req-23769934-c11b-479e-8d62-87be2cdc3286'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:28.582102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d897789ca8446eaa73a4c4f6e7d677', 'request_id': 'req-23769934-c11b-479e-8d62-87be2cdc3286'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:28.590818 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d897789ca8446eaa73a4c4f6e7d677', 'request_id': 'req-23769934-c11b-479e-8d62-87be2cdc3286'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:28.617934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] The token cache is being invalidate because project 85d897789ca8446eaa73a4c4f6e7d677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:28.618315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:28.619444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-23769934-c11b-479e-8d62-87be2cdc3286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:28.622530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1246/2492] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:28 2022] DELETE /identity/v3/projects/85d897789ca8446eaa73a4c4f6e7d677 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:28.628964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:28.629162 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:28.629480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:28.651716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:28.653033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:28.742283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1247/2493] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:52:28.750767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:28.753529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:28.753529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:28.778676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:28.780365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:28.789217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1247/2494] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:28.794194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:28.805366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb52188-8ba5-4c30-8b17-76a74aa896d3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:28.811019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:28.811674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:28.811951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:28.812352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:28.813473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:28.813473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:28.814964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:28.832389 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:28.836051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1248/2495] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:52:28.843272 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99d359-d59d-46a8-9d05-8e423114647d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:28.849062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:28.849880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:28.850235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:28.850577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:28.851317 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:28.853476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:28.897153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1248/2496] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:28 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:28.901339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:28.912149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ede3-d787-4ecd-a1b4-ba92a9675030 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.000231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.000843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.001244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.001520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.002168 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.004271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.043385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1249/2497] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:28 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:29.045007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.057452 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1db4f0-b9d4-4fcf-8e6e-40e105577a80 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.062816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.063456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.063744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.064105 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.064763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.066438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.085522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1249/2498] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:29.091305 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.102739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6852b11-622d-41a3-8db3-5c37d2927f2d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.108457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b6ecc-10d2-487f-a75a-39b2a1870eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.109130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b6ecc-10d2-487f-a75a-39b2a1870eed admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.109369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b6ecc-10d2-487f-a75a-39b2a1870eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.109635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b6ecc-10d2-487f-a75a-39b2a1870eed admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.110330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b6ecc-10d2-487f-a75a-39b2a1870eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.112223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b6ecc-10d2-487f-a75a-39b2a1870eed admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.123937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1250/2499] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:29.128114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.138215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89e791-da9f-478a-b6f7-358a0ceb9344 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.142833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.143517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.143800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.144080 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] PATH_INFO: `/v3/projects/3c4eb8c79e1b451292edf5c59af5feca/users/9afa99712d3346c2855bc79cb72ee4b3/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.158790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4eb8c79e1b451292edf5c59af5feca, user_id=9afa99712d3346c2855bc79cb72ee4b3, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.161561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.175406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1250/2500] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:29 2022] PUT /identity/v3/projects/3c4eb8c79e1b451292edf5c59af5feca/users/9afa99712d3346c2855bc79cb72ee4b3/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:29.187766 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.188020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.188298 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.217283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] MFA Rules not processed for user `9afa99712d3346c2855bc79cb72ee4b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:29.218504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:29.243174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:29.286438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1251/2501] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:29.291128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.298538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:29.358455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23627ae-de61-40fe-b021-11108106b0c9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.363168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.363800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.364078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.364347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.364993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.367020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.400265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1251/2502] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:29 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:29.405059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.416211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b121c8-d232-40ed-b858-bb1f9680353c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.476187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.476801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.477079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.477347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.477987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.479949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.502837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1252/2503] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:29 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:29.506916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.518019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f36f61-9921-4e85-a086-befc30a5c5d0 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.522713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.523319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.523618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.523887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.524526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.526170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.537379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1252/2504] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:29.546050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.557649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bce45-3fb2-474d-a4e7-32144d1b3f65 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.562901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.563634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.563952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.564222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] PATH_INFO: `/v3/projects/91cf630022d0481fa4a358195dcc6995/users/6a2d990404214dd0840f3c98443babf9/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.580225 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cf630022d0481fa4a358195dcc6995, user_id=6a2d990404214dd0840f3c98443babf9, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.582938 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.597972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1253/2505] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:29 2022] PUT /identity/v3/projects/91cf630022d0481fa4a358195dcc6995/users/6a2d990404214dd0840f3c98443babf9/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:29.602228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.613829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5e692-7bd8-40a4-9905-b91371569080 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.677162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e0069-2ce7-4997-8fc3-25c4602c6871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.677725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e0069-2ce7-4997-8fc3-25c4602c6871 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.677997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e0069-2ce7-4997-8fc3-25c4602c6871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.678293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e0069-2ce7-4997-8fc3-25c4602c6871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.678949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e0069-2ce7-4997-8fc3-25c4602c6871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.680569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e0069-2ce7-4997-8fc3-25c4602c6871 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.690777 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1253/2506] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:29.695014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.706276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8e11-6a3f-419e-b563-a2f9e00e9ce7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.711336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.712067 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.712350 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.712612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] PATH_INFO: `/v3/projects/91cf630022d0481fa4a358195dcc6995/users/6a2d990404214dd0840f3c98443babf9/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.714531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cf630022d0481fa4a358195dcc6995, user_id=6a2d990404214dd0840f3c98443babf9, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.718054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.728479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1254/2507] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:29 2022] PUT /identity/v3/projects/91cf630022d0481fa4a358195dcc6995/users/6a2d990404214dd0840f3c98443babf9/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:29.739213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.739526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.739797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.768495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] MFA Rules not processed for user `6a2d990404214dd0840f3c98443babf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:29.769703 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:29.836705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:29.841009 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1254/2508] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:29.852499 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:29.854477 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:29.914998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abd2b7-061f-4c50-bf9e-bb9e5e51a4b8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:29.919207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:29.920108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:29.920384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:29.920653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:29.930117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:29.931541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:29.942258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1255/2509] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:30.022149 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:30.033832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333755b-dee7-4854-907a-8dd7b6ab5448 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:30.038712 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:30.039322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:30.039627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:30.040023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:30.104733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:30.107026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:30.118907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1255/2510] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:30.150928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:30.152976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:30.181424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:30.228539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca94e8e-100d-4528-a7cc-9cc778d11890 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:30.233071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:30.233691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:30.233970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:30.234239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:30.236308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:30.327031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:30.328503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:30.338656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1256/2511] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:33.876974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:33.879278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:33.895405 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d51f6-ecbe-4e25-88c6-cdd039e1ff1e service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:33.900851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:33.901560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:33.901896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:33.902759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] PATH_INFO: `/v3/users/6974a4c8fffd4bc2933e754770cb912e` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:33.904511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6974a4c8fffd4bc2933e754770cb912e)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:33.906658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:33.968698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6974a4c8fffd4bc2933e754770cb912e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:33.977941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6974a4c8fffd4bc2933e754770cb912e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:33.978629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6974a4c8fffd4bc2933e754770cb912e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:33.979231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6974a4c8fffd4bc2933e754770cb912e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:33.987058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9f7faf23-b259-461c-a271-1460f310973f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6974a4c8fffd4bc2933e754770cb912e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:34.005190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1256/2512] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:33 2022] DELETE /identity/v3/users/6974a4c8fffd4bc2933e754770cb912e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:34.179241 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.189954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576200a-0159-4083-a8ab-31d096ed7b6c service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.249686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.250320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.250594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.250861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] PATH_INFO: `/v3/projects/3417cf79d7d748d483e8aeb8289b75f6` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.252007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3417cf79d7d748d483e8aeb8289b75f6)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.253916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.300965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3417cf79d7d748d483e8aeb8289b75f6', 'request_id': 'req-da50840d-d504-4ea2-9d9f-5bda3611b51e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:34.309173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3417cf79d7d748d483e8aeb8289b75f6', 'request_id': 'req-da50840d-d504-4ea2-9d9f-5bda3611b51e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:34.315444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3417cf79d7d748d483e8aeb8289b75f6', 'request_id': 'req-da50840d-d504-4ea2-9d9f-5bda3611b51e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:34.323113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:34.328748 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] The token cache is being invalidate because project 3417cf79d7d748d483e8aeb8289b75f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:34.329189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:34.330346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:34.331965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1257/2513] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:34 2022] DELETE /identity/v3/projects/3417cf79d7d748d483e8aeb8289b75f6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:34.337043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5b3b-b872-46a6-ab65-4bd5cf54c96e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.337326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5b3b-b872-46a6-ab65-4bd5cf54c96e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.337596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5b3b-b872-46a6-ab65-4bd5cf54c96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.357870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-dead5b3b-b872-46a6-ab65-4bd5cf54c96e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:34.359064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dead5b3b-b872-46a6-ab65-4bd5cf54c96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:34.395913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.397479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:34.422167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1257/2514] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:34.430163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.430443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.430714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.450604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:34.451852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:34.459702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1258/2515] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:52:34.463835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.464005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50840d-d504-4ea2-9d9f-5bda3611b51e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.468400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.469557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.469843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.470050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:34.470176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.472049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:34.474442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899c77a-c2a2-495e-b9f9-3288c412712c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.481066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.481976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.482588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.483504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.484174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:34.484496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.486155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.498210 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1259/2516] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:52:34.502697 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.512847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519ae0e-6129-4b0b-8611-afcd7c0aca59 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.517603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.518191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.518463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.518728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.519352 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.521276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.533816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.535289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.547346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1258/2517] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:34.551562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1260/2518] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:34 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:34.555796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.568775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a295db-1e85-42de-8090-ea2e72e47cfd service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.634461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.635094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.635374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.635734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.636460 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.639149 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.665354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1259/2519] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:34 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:34.666929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.678061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed2d35-e36b-4ae6-bd78-f58818efc0b1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.682956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.684152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.684428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.684790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.685466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.687133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.698532 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1261/2520] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:34.703401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.714749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34a3f2-13f8-4dc8-926c-3e17cfb3b53e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.720318 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa31a50-7f8f-4f3c-84af-5c74e2c4ac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.720945 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa31a50-7f8f-4f3c-84af-5c74e2c4ac56 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.721223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa31a50-7f8f-4f3c-84af-5c74e2c4ac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.721489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa31a50-7f8f-4f3c-84af-5c74e2c4ac56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.722744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa31a50-7f8f-4f3c-84af-5c74e2c4ac56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.724022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa31a50-7f8f-4f3c-84af-5c74e2c4ac56 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.735763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1260/2521] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:34.740089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.751800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cc2f7-0231-49ba-be78-a27afccab0f3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:34.756893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:34.757738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.758022 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.758289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] PATH_INFO: `/v3/projects/19addd8250b744629ce432f2c47777da/users/229255d3e5d94652a05a2a5f3a7f8c6b/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.776676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19addd8250b744629ce432f2c47777da, user_id=229255d3e5d94652a05a2a5f3a7f8c6b, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:34.779670 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:34.795192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1262/2522] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:34 2022] PUT /identity/v3/projects/19addd8250b744629ce432f2c47777da/users/229255d3e5d94652a05a2a5f3a7f8c6b/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:34.805816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:34.806198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:34.806563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:34.839387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] MFA Rules not processed for user `229255d3e5d94652a05a2a5f3a7f8c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:34.841229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:34.866807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:34.919037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1261/2523] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:34.931010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:34.932723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:35.015580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74612531-9abe-4c67-818d-c65e7adc73a8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:35.021907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74c8f9-747a-46e3-8352-8c7ee6976ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:35.022538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74c8f9-747a-46e3-8352-8c7ee6976ab7 service nova] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:35.022816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74c8f9-747a-46e3-8352-8c7ee6976ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:35.023088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74c8f9-747a-46e3-8352-8c7ee6976ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:35.033586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74c8f9-747a-46e3-8352-8c7ee6976ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:35.035243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74c8f9-747a-46e3-8352-8c7ee6976ab7 service nova] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:35.047945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1263/2524] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:42.971501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:42.973096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:43.040005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6172-64cc-495c-b5cf-d0bbc8684737 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:43.045761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:43.046397 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:43.046677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:43.046952 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] PATH_INFO: `/v3/users/e2484143c56d4629a5466f5504d2e869` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:43.048113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2484143c56d4629a5466f5504d2e869)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:43.050049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:43.107049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2484143c56d4629a5466f5504d2e869'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.130126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2484143c56d4629a5466f5504d2e869'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.140234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2484143c56d4629a5466f5504d2e869'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.140861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2484143c56d4629a5466f5504d2e869'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.150942 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2484143c56d4629a5466f5504d2e869'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.157846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1262/2525] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:42 2022] DELETE /identity/v3/users/e2484143c56d4629a5466f5504d2e869 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:43.163033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:43.163665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:43.163966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:43.184965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:43.186398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:43.250613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1264/2526] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:43.260886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:43.263139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:43.313531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:43.327693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a29549-d81c-480d-bcc4-d5ee12c00b1b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:43.331365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:43.332004 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:43.332282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:43.332549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:43.342541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:43.344016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:43.353708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1263/2527] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:43.655582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:43.657176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:43.671243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1400f51-33b1-4bbe-a026-6c606941d8bf None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:43.676115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:43.676745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:43.677016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:43.677865 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] PATH_INFO: `/v3/projects/3dc2cb99fd7d45479529ba277e5d1321` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:43.679148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc2cb99fd7d45479529ba277e5d1321)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:43.681174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:43.698047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:43.734145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc2cb99fd7d45479529ba277e5d1321', 'request_id': 'req-cdacdcf5-a42f-4faf-a065-7da0738122ad'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.740803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc2cb99fd7d45479529ba277e5d1321', 'request_id': 'req-cdacdcf5-a42f-4faf-a065-7da0738122ad'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.747250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc2cb99fd7d45479529ba277e5d1321', 'request_id': 'req-cdacdcf5-a42f-4faf-a065-7da0738122ad'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.758411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] The token cache is being invalidate because project 3dc2cb99fd7d45479529ba277e5d1321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:43.758621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.759313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.760851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1265/2528] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:43 2022] DELETE /identity/v3/projects/3dc2cb99fd7d45479529ba277e5d1321 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:43.765636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:43.775258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8599630-3c2b-49b9-be24-3bb9249df84a service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:43.832420 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:43.833041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:43.833323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:43.833612 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] PATH_INFO: `/v3/users/50ef4943b58f4cbb8145756a3420500a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:43.834706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ef4943b58f4cbb8145756a3420500a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:43.836641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:43.884856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ef4943b58f4cbb8145756a3420500a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.898570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ef4943b58f4cbb8145756a3420500a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.908984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ef4943b58f4cbb8145756a3420500a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.909871 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ef4943b58f4cbb8145756a3420500a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.918213 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-246594f0-d98b-4dc3-96e6-f355bb7f04f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ef4943b58f4cbb8145756a3420500a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:43.924556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1264/2529] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:43 2022] DELETE /identity/v3/users/50ef4943b58f4cbb8145756a3420500a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:44.198203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.200042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:44.228763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacdcf5-a42f-4faf-a065-7da0738122ad admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.286000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.286546 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.286819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.287086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] PATH_INFO: `/v3/projects/6a8a897925a6482781e53bfdbfb2b543` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.288223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8a897925a6482781e53bfdbfb2b543)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.290146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.335219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8a897925a6482781e53bfdbfb2b543', 'request_id': 'req-c94448e1-1512-4514-a7f3-30e47da12439'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:44.343396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8a897925a6482781e53bfdbfb2b543', 'request_id': 'req-c94448e1-1512-4514-a7f3-30e47da12439'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:44.349163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8a897925a6482781e53bfdbfb2b543', 'request_id': 'req-c94448e1-1512-4514-a7f3-30e47da12439'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:44.360653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] The token cache is being invalidate because project 6a8a897925a6482781e53bfdbfb2b543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:44.360935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:44.361584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c94448e1-1512-4514-a7f3-30e47da12439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:44.363084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1266/2530] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:44 2022] DELETE /identity/v3/projects/6a8a897925a6482781e53bfdbfb2b543 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:44.368082 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.368574 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.368712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.388412 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:44.389650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:44.448402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1265/2531] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:44.457539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.457824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.458097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.478110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:44.479306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:44.487193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1267/2532] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:44.491480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.497932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:44.501846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1c3e-5165-4a10-8064-ec060b1b5386 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.507481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.508091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.508733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.509147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.509905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:44.510379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.513131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.524576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1266/2533] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:44.529158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.538238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6ccc0-639b-4300-b925-746606ebcb47 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.542900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.543382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.543686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.543950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.544821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.546747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.574162 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1268/2534] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:44 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:44.578192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.587893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcf35f-ffc3-47bb-992b-cac66f2abc08 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.645393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.645996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.646274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.646539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.647196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.649135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.693505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1267/2535] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:44 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:44.697340 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.702364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:44.750757 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3ca3-abd9-4f6e-a8c8-4765b906c89f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.755676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.756331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.756636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.756905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.757568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.759163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.769978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1269/2536] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:44.774244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.830347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a3f8b-6bbb-4745-8c30-5b763c9b13a1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.835166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe3af2-fb73-4668-a192-5b59687cc5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.835798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe3af2-fb73-4668-a192-5b59687cc5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.836303 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe3af2-fb73-4668-a192-5b59687cc5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.836576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe3af2-fb73-4668-a192-5b59687cc5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.837205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe3af2-fb73-4668-a192-5b59687cc5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.838909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe3af2-fb73-4668-a192-5b59687cc5e0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.848767 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1268/2537] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:44.853053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:44.864156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db1e7-be79-4168-b33f-00dfd998c7fa admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:44.868800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:44.869468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.869727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.869997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] PATH_INFO: `/v3/projects/bf898e26fa1241ba8e236f4c5069fce0/users/f6d9ed3175fd43b58118174548c0c69c/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.885892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf898e26fa1241ba8e236f4c5069fce0, user_id=f6d9ed3175fd43b58118174548c0c69c, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:44.888628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:44.901551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1270/2538] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:44 2022] PUT /identity/v3/projects/bf898e26fa1241ba8e236f4c5069fce0/users/f6d9ed3175fd43b58118174548c0c69c/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:44.912905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:44.913192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:44.913469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:44.942877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] MFA Rules not processed for user `f6d9ed3175fd43b58118174548c0c69c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:44.944164 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:45.016041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:45.020778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1269/2539] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:45.025346 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.097411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494a2ab-bb45-4a05-aaad-cf19b893bbe6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.102207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.102821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.103100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.103368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.104036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.105983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.132326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1271/2540] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:45 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:45.137114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.147770 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d686ba4-158d-4185-acec-31f92f9989fb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.204194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.204791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.205061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.205328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.205973 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.207917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.232704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1270/2541] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:45 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:45.233929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.245281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb688-536f-4b9a-a13b-d4a37a06d59a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.251781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.252401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.252715 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.253002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.254192 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.255844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.272684 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1272/2542] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:45.273866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.286380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b72d-85c1-4450-bd54-4eac764cfbdb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.291143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.299509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.299509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.299509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] PATH_INFO: `/v3/projects/16c696a6c9244241967a75eca14b10e5/users/737d46f71984476aa3e0e801b22ad6a8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.308678 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c696a6c9244241967a75eca14b10e5, user_id=737d46f71984476aa3e0e801b22ad6a8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.311798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.325293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1271/2543] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:45 2022] PUT /identity/v3/projects/16c696a6c9244241967a75eca14b10e5/users/737d46f71984476aa3e0e801b22ad6a8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:45.329238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.339782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53bc3b-4b77-4371-af51-7824023a9a27 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.401876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ebf1-6113-456e-a2e4-bdab15bedde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.402614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ebf1-6113-456e-a2e4-bdab15bedde5 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.402959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ebf1-6113-456e-a2e4-bdab15bedde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.403200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ebf1-6113-456e-a2e4-bdab15bedde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.403878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ebf1-6113-456e-a2e4-bdab15bedde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.405541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ebf1-6113-456e-a2e4-bdab15bedde5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.411215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:45.416706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1273/2544] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:45.421763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.432191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880532-5cde-4740-8db8-772259c9857b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.438789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.439490 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.439773 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.440053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] PATH_INFO: `/v3/projects/16c696a6c9244241967a75eca14b10e5/users/737d46f71984476aa3e0e801b22ad6a8/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.442044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c696a6c9244241967a75eca14b10e5, user_id=737d46f71984476aa3e0e801b22ad6a8, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.444793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.460556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1272/2545] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:45 2022] PUT /identity/v3/projects/16c696a6c9244241967a75eca14b10e5/users/737d46f71984476aa3e0e801b22ad6a8/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:45.476960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.477335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.477612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.506900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] MFA Rules not processed for user `737d46f71984476aa3e0e801b22ad6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:45.508195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:45.580868 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1274/2546] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:52:45.593209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.595106 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:45.681321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926dfd-1495-461f-af04-1589cc0075a6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.685378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.686057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.686386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.686653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.698384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.700740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.712510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1273/2547] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:45.859108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.872987 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3903b-1b81-47a2-8eb8-f65efbfe6de1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.883691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6971d5-59bd-41dd-a2a6-88d103b9209c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.886874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:45.889043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6971d5-59bd-41dd-a2a6-88d103b9209c service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.889043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6971d5-59bd-41dd-a2a6-88d103b9209c service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.889361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:45.889523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6971d5-59bd-41dd-a2a6-88d103b9209c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.946170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:45.985700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ac95e-1341-4a55-9f94-6a2bd9a5981b service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:45.992530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:45.993753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:45.994057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6971d5-59bd-41dd-a2a6-88d103b9209c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:45.994279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:45.994649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] PATH_INFO: `/v3/users/229255d3e5d94652a05a2a5f3a7f8c6b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:45.996816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6971d5-59bd-41dd-a2a6-88d103b9209c service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:45.997602 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229255d3e5d94652a05a2a5f3a7f8c6b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:46.000479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:46.013218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1275/2548] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:46.073429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229255d3e5d94652a05a2a5f3a7f8c6b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.129536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229255d3e5d94652a05a2a5f3a7f8c6b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.145278 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229255d3e5d94652a05a2a5f3a7f8c6b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.146260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229255d3e5d94652a05a2a5f3a7f8c6b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.165227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229255d3e5d94652a05a2a5f3a7f8c6b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.179105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1274/2549] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:45 2022] DELETE /identity/v3/users/229255d3e5d94652a05a2a5f3a7f8c6b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:46.184306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:46.184609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:46.184837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:46.266654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:46.268965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:46.379392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1276/2550] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:46.390867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:46.392739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:46.464327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219f3b-e401-4b5c-8dd0-015d257f89d6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:46.473775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb41e7e-6729-40b7-9e9e-f034649940a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:46.474398 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41e7e-6729-40b7-9e9e-f034649940a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:46.474673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41e7e-6729-40b7-9e9e-f034649940a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:46.474936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41e7e-6729-40b7-9e9e-f034649940a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:46.484438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb41e7e-6729-40b7-9e9e-f034649940a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:46.485834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb41e7e-6729-40b7-9e9e-f034649940a8 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:46.497839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1275/2551] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:46.800537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:46.802029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:46.815582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e6843-b92f-4e1c-bcec-93cf6a92c76c None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:46.820451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:46.821166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:46.821449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:46.821723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] PATH_INFO: `/v3/projects/19addd8250b744629ce432f2c47777da` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:46.822915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19addd8250b744629ce432f2c47777da)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:46.826062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:46.839058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:46.879673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19addd8250b744629ce432f2c47777da', 'request_id': 'req-b3b47220-03a9-42ca-8139-f56d4960d28b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.887838 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19addd8250b744629ce432f2c47777da', 'request_id': 'req-b3b47220-03a9-42ca-8139-f56d4960d28b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.894121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19addd8250b744629ce432f2c47777da', 'request_id': 'req-b3b47220-03a9-42ca-8139-f56d4960d28b'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.906839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] The token cache is being invalidate because project 19addd8250b744629ce432f2c47777da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:46.907158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.907853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b3b47220-03a9-42ca-8139-f56d4960d28b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:46.909379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1277/2552] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:46 2022] DELETE /identity/v3/projects/19addd8250b744629ce432f2c47777da => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:46.914521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:46.914811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:46.915084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:46.968720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:46.969970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:47.005858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:47.044194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1276/2553] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:47.052624 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.052909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.053184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.080084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:47.081364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:47.092881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1278/2554] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:47.096405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.107245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169a8c1-138c-4a5a-9ea5-0c2e412cfbd8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.111996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.112642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.112947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.113220 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.113870 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:47.114172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.115816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.128779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1277/2555] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:47.133197 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.142593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2eeb63-f09a-4ffc-8c9c-d22fff8f9638 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.147232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.147866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.148138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.148409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.149046 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.150978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.179751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1279/2556] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:47 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:47.184981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.195053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5006-cbcd-4c50-817a-1ba844c22695 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.256005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.256637 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.256917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.257203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.257828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.259844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.287515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1278/2557] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:47 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:47.288761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.301694 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e96c9-73b1-4261-9d2c-e4be7a846cf5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.306581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.307199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.307510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.307907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.310885 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.311785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.326042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1280/2558] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:47.327233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.344140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c53a7-e166-49e7-bbc1-d3290c478a4f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.349435 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6959794-99f3-4fd2-90fa-c5a678984a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.350258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6959794-99f3-4fd2-90fa-c5a678984a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.350439 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6959794-99f3-4fd2-90fa-c5a678984a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.350650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6959794-99f3-4fd2-90fa-c5a678984a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.351307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6959794-99f3-4fd2-90fa-c5a678984a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.353090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6959794-99f3-4fd2-90fa-c5a678984a02 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.370903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1279/2559] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:47.372155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.383632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c728e-fc01-43bb-90d1-413a31366ab9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.395833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.396635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.397140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.397424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] PATH_INFO: `/v3/projects/2b270b31070d4e82808a391b22a5256a/users/06147bc69c8f4677bdfb82a87f91e75e/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.416295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b270b31070d4e82808a391b22a5256a, user_id=06147bc69c8f4677bdfb82a87f91e75e, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.419883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.434558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1281/2560] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:47 2022] PUT /identity/v3/projects/2b270b31070d4e82808a391b22a5256a/users/06147bc69c8f4677bdfb82a87f91e75e/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:47.440503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.440858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.441353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.463722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:47.466066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:47.564066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1280/2561] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:47.577308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.579414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:47.621695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:47.623182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:47.690143 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:47.703454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2943d1-9e14-4b84-8146-71014e587794 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.704813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde6f56-411a-43c8-bf56-6d3146b6449c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:47.708118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5c657-9b4e-44a2-99ed-89fab26cf1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.708739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5c657-9b4e-44a2-99ed-89fab26cf1ea service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.709061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5c657-9b4e-44a2-99ed-89fab26cf1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.709548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5c657-9b4e-44a2-99ed-89fab26cf1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.711309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:47.712044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:47.712447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:47.712725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:47.715199 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:47.722353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5c657-9b4e-44a2-99ed-89fab26cf1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.723946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5c657-9b4e-44a2-99ed-89fab26cf1ea service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.731268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:47.733959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:47.738261 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1282/2562] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:47.746445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1281/2563] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:49.111360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.111866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.111965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.148980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] MFA Rules not processed for user `06147bc69c8f4677bdfb82a87f91e75e`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:49.150817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:49.192644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:49.241663 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1283/2564] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:49 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:49.245961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:49.257090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093be1e-7c52-458f-b1d2-537d9722a50f service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:49.278390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:49.279165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.279954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.280394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.281093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:49.286520 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:49.332488 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1282/2565] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:49 2022] POST /identity/v3/projects => generated 401 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:49.336344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:49.349995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869e76a-ce2f-409e-b343-875943a1ca8d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:49.430039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:49.430717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.431025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.431303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.432113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:49.434779 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:49.495432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1284/2566] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:49 2022] POST /identity/v3/users => generated 307 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:49.499032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:49.512258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35865b3-7e25-4a41-9118-49fdfdadd990 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:49.519126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:49.519944 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.520335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.520704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.521543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:49.524359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:49.538911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1283/2567] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:52:49.543873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:49.555765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15859-75a6-4fc7-bdcb-cc9052ec9c79 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:49.562662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:49.563399 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.563739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.564032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] PATH_INFO: `/v3/projects/12dc227ca692444ab8b57b89f995b0fc/users/ee3e244e75db41239509bb1d0544731c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.582356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12dc227ca692444ab8b57b89f995b0fc, user_id=ee3e244e75db41239509bb1d0544731c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:49.585825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:49.602087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1285/2568] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:49 2022] PUT /identity/v3/projects/12dc227ca692444ab8b57b89f995b0fc/users/ee3e244e75db41239509bb1d0544731c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:49.606629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:49.616802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16a8aa-de1d-4621-b2fa-1977c1e389b7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:49.676591 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14276f3b-d18e-4dff-9d7d-7962b8d55272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:49.677631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14276f3b-d18e-4dff-9d7d-7962b8d55272 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.678154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14276f3b-d18e-4dff-9d7d-7962b8d55272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.678658 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14276f3b-d18e-4dff-9d7d-7962b8d55272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.679913 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14276f3b-d18e-4dff-9d7d-7962b8d55272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:49.683540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14276f3b-d18e-4dff-9d7d-7962b8d55272 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:49.695890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1284/2569] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:49.701190 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:49.712601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f6878-f572-410b-b6ac-0b834772214e admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:49.723472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:49.724157 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:49.724431 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:49.724700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] PATH_INFO: `/v3/projects/12dc227ca692444ab8b57b89f995b0fc/users/ee3e244e75db41239509bb1d0544731c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:49.726920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=12dc227ca692444ab8b57b89f995b0fc, user_id=ee3e244e75db41239509bb1d0544731c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:49.729596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:49.740666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1286/2570] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:49 2022] PUT /identity/v3/projects/12dc227ca692444ab8b57b89f995b0fc/users/ee3e244e75db41239509bb1d0544731c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:50.889798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:50.890579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:50.890579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:50.964929 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] MFA Rules not processed for user `ee3e244e75db41239509bb1d0544731c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:50.967469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:51.078417 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1285/2571] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:50 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:52.476800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:52.478834 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:52.575978 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:52.616211 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:52.621494 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe111-e0cb-4e47-a9aa-ede482319bcd admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:52.628914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:52.634266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:52.635518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:52.637403 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:52.638003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:52.640766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:52.657130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:52.658815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:52.672951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1287/2572] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:52.675527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9424af-5097-44ee-a342-309040c9c8e9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:52.683924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:52.684711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:52.685007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:52.685752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] PATH_INFO: `/v3/users/06147bc69c8f4677bdfb82a87f91e75e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:52.687391 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06147bc69c8f4677bdfb82a87f91e75e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:52.690652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:52.756509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06147bc69c8f4677bdfb82a87f91e75e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:52.780378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06147bc69c8f4677bdfb82a87f91e75e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:52.789723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06147bc69c8f4677bdfb82a87f91e75e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:52.790449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06147bc69c8f4677bdfb82a87f91e75e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:52.800430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06147bc69c8f4677bdfb82a87f91e75e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:52.811653 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1286/2573] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:52 2022] DELETE /identity/v3/users/06147bc69c8f4677bdfb82a87f91e75e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:53.029725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:53.040717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3a727-ac15-455d-a791-e587724074c8 service placement] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:53.120221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:53.121509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:53.121792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:53.122091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] PATH_INFO: `/v3/projects/2b270b31070d4e82808a391b22a5256a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:53.124176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b270b31070d4e82808a391b22a5256a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:53.127661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:53.190639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b270b31070d4e82808a391b22a5256a', 'request_id': 'req-310e7329-e995-42b6-a87c-db07a11fd640'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.200222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b270b31070d4e82808a391b22a5256a', 'request_id': 'req-310e7329-e995-42b6-a87c-db07a11fd640'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.208722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b270b31070d4e82808a391b22a5256a', 'request_id': 'req-310e7329-e995-42b6-a87c-db07a11fd640'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.222994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] The token cache is being invalidate because project 2b270b31070d4e82808a391b22a5256a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:53.223281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.224043 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.225698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1288/2574] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:53 2022] DELETE /identity/v3/projects/2b270b31070d4e82808a391b22a5256a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:53.230313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:53.231919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:53.253744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47d88-e96d-4e5b-8c79-9728f93ed2a2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:53.332446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:53.333094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:53.333376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:53.333651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] PATH_INFO: `/v3/users/ee3e244e75db41239509bb1d0544731c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:53.334821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3e244e75db41239509bb1d0544731c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:53.336806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:53.397216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3e244e75db41239509bb1d0544731c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.407452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:53.413885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3e244e75db41239509bb1d0544731c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.426413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3e244e75db41239509bb1d0544731c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.427328 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3e244e75db41239509bb1d0544731c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.437315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08f54a9a-e672-4337-b19b-8e17cdbedf54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3e244e75db41239509bb1d0544731c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.443866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1287/2575] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:53 2022] DELETE /identity/v3/users/ee3e244e75db41239509bb1d0544731c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:53.704032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:53.705817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:53.729531 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7329-e995-42b6-a87c-db07a11fd640 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:53.851471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:53.852205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:53.855574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:53.855574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] PATH_INFO: `/v3/projects/12dc227ca692444ab8b57b89f995b0fc` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:53.857963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12dc227ca692444ab8b57b89f995b0fc)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:53.860096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:53.896024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:53.931955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12dc227ca692444ab8b57b89f995b0fc', 'request_id': 'req-8a9fca44-9797-409c-8de0-2fe10ca67df7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.943746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12dc227ca692444ab8b57b89f995b0fc', 'request_id': 'req-8a9fca44-9797-409c-8de0-2fe10ca67df7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.951374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12dc227ca692444ab8b57b89f995b0fc', 'request_id': 'req-8a9fca44-9797-409c-8de0-2fe10ca67df7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.968961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] The token cache is being invalidate because project 12dc227ca692444ab8b57b89f995b0fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:53.969279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.969938 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a9fca44-9797-409c-8de0-2fe10ca67df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:53.971789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1289/2576] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:53 2022] DELETE /identity/v3/projects/12dc227ca692444ab8b57b89f995b0fc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:53.979338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144d19-f078-46fc-9b39-043115f5715d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:53.979338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144d19-f078-46fc-9b39-043115f5715d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:53.979338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144d19-f078-46fc-9b39-043115f5715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.001448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-19144d19-f078-46fc-9b39-043115f5715d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:54.002700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-19144d19-f078-46fc-9b39-043115f5715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:54.095976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1288/2577] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:54.107814 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.108421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.108807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.132713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:54.134120 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:54.147091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1290/2578] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:54.152345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19144d19-f078-46fc-9b39-043115f5715d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.165729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19144d19-f078-46fc-9b39-043115f5715d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.177266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.177909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.178261 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.178449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.179112 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:54.179707 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.181486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.203078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1289/2579] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:54.209020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.224128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a375d-e67d-4c1a-9988-efd523c55fbe None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.228680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.229727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.230434 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.231410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.232335 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.235078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.288023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1291/2580] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:54 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:54.294899 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.305444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64748c-0e99-4ef6-a1c5-262437617243 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.384235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.384906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.385235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.385509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.387245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.390790 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.433633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1290/2581] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:54 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:54.438077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.454529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a18be-7f0a-488d-837d-eae20a0c55a7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.463005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.463874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.465184 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.465678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.466601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.469320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.483481 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1292/2582] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:54.486464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.499708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce399e-a549-49a3-9616-1039e27c46ab admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.505471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83dafd-ed90-47b5-bfa9-859a923d050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.506079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83dafd-ed90-47b5-bfa9-859a923d050a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.506540 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83dafd-ed90-47b5-bfa9-859a923d050a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.506906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83dafd-ed90-47b5-bfa9-859a923d050a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.507682 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83dafd-ed90-47b5-bfa9-859a923d050a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.509746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83dafd-ed90-47b5-bfa9-859a923d050a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.522873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1291/2583] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:54.528493 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.545594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad2854-5b52-4d1f-92cd-2046af837d58 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.555222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.556314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.556797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.557221 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] PATH_INFO: `/v3/projects/acc43c9afbc443e59b8aae4fc491cfae/users/4dc16ee508b3449ea9db4883b033045f/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.587905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc43c9afbc443e59b8aae4fc491cfae, user_id=4dc16ee508b3449ea9db4883b033045f, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.594051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.618361 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1293/2584] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:54 2022] PUT /identity/v3/projects/acc43c9afbc443e59b8aae4fc491cfae/users/4dc16ee508b3449ea9db4883b033045f/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:54.630212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.630636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.631040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.666339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] MFA Rules not processed for user `4dc16ee508b3449ea9db4883b033045f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:54.668156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:54.702474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:54.767889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1292/2585] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:54.773087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.873306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943545f-a3fe-4d73-86ac-4a4d0e3e3306 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:54.879743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:54.880596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:54.880971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:54.881344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:54.882131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:54.884528 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:54.930719 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1294/2586] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:54 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:54.935937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:54.951872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e82a05-de75-48a0-b023-ff68a5a5c41c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:55.036742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:55.038177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.038996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.039641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.040625 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:55.042877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:55.117492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1293/2587] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:54 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:55.118714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:55.140104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5203a0-c97a-4618-a704-66b2dfbd5ea2 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:55.150036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:55.150860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.151946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.151946 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.152573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:55.154503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:55.171487 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1295/2588] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:55.172890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:55.185531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f99d0-bdc0-40cc-a2dc-56e1b131c274 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:55.192860 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:55.193898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.194367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.194785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] PATH_INFO: `/v3/projects/224952a5828b412291619f5f8cbd795f/users/c1212f061c49489eb79aa5563fa55f52/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.219442 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224952a5828b412291619f5f8cbd795f, user_id=c1212f061c49489eb79aa5563fa55f52, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:55.222894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:55.243160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1294/2589] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:55 2022] PUT /identity/v3/projects/224952a5828b412291619f5f8cbd795f/users/c1212f061c49489eb79aa5563fa55f52/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:55.247927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:55.265060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730e056-9a88-45b8-9f9c-ccefbcc7c8cb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:55.309558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:55.374135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5b299-635b-435a-9eda-90eb2ae6bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:55.374924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b299-635b-435a-9eda-90eb2ae6bce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.375510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b299-635b-435a-9eda-90eb2ae6bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.375871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b299-635b-435a-9eda-90eb2ae6bce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.377172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b299-635b-435a-9eda-90eb2ae6bce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:55.379057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b299-635b-435a-9eda-90eb2ae6bce8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:55.398424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1296/2590] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:55.399514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:55.412333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea69969-1321-491a-b4ad-b2240bc09f79 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:55.418673 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:55.420049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.420299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.420657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] PATH_INFO: `/v3/projects/224952a5828b412291619f5f8cbd795f/users/c1212f061c49489eb79aa5563fa55f52/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.423754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224952a5828b412291619f5f8cbd795f, user_id=c1212f061c49489eb79aa5563fa55f52, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:55.426491 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:55.452116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1295/2591] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:55 2022] PUT /identity/v3/projects/224952a5828b412291619f5f8cbd795f/users/c1212f061c49489eb79aa5563fa55f52/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:55.454373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.454705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.455013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.495066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] MFA Rules not processed for user `c1212f061c49489eb79aa5563fa55f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:55.497800 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:55.596890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1297/2592] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:55.628302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:55.630076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:55.760217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caf1be-d607-42a2-bb42-14a2b0a14ef9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:55.766131 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:55.766931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:55.767206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:55.767517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:55.796551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:55.798179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:55.822046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1296/2593] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:56.379382 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:56.396056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256f48e-fd9b-40c5-8d72-d14a037a5a21 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:56.405232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1212f061c49489eb79aa5563fa55f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '224952a5828b412291619f5f8cbd795f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:56.407394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:56.407726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:56.408005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:56.408691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:56.408998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:56.411571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:56.438662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1298/2594] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:56.439966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:56.452033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfa1e-143a-441c-ac9b-d365d9edac90 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:56.463031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1212f061c49489eb79aa5563fa55f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '224952a5828b412291619f5f8cbd795f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:56.463833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:56.465294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:56.466471 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:56.468911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:56.474539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:56.562430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1297/2595] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:56 2022] POST /identity/v3/projects => generated 355 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:56.567409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:56.585387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7a01d-1143-4566-b3cc-bfe8422c3950 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:56.715655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1212f061c49489eb79aa5563fa55f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '224952a5828b412291619f5f8cbd795f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:56.716792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:56.717279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:56.718234 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:56.719108 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:56.720279 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:56.724130 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:56.748630 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1299/2596] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:56.830779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:56.843691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:56.854530 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81d2b4-29ae-460d-a7f5-7fa9375d571e tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:56.870564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1212f061c49489eb79aa5563fa55f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '224952a5828b412291619f5f8cbd795f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:56.871577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:56.871923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:56.872203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] PATH_INFO: `/v3/projects/300f69c191ef469eaf11c18e210ec6bc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:56.876999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorizing `identity:delete_project(project_id=300f69c191ef469eaf11c18e210ec6bc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:56.881986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:56.955684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '300f69c191ef469eaf11c18e210ec6bc', 'request_id': 'req-f2e13200-2c05-4b1a-a720-92765ac47f26'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:56.969013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '300f69c191ef469eaf11c18e210ec6bc', 'request_id': 'req-f2e13200-2c05-4b1a-a720-92765ac47f26'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:56.975761 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '300f69c191ef469eaf11c18e210ec6bc', 'request_id': 'req-f2e13200-2c05-4b1a-a720-92765ac47f26'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:56.989698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] The token cache is being invalidate because project 300f69c191ef469eaf11c18e210ec6bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:56.990020 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:56.990787 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:56.993081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1298/2597] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:56 2022] DELETE /identity/v3/projects/300f69c191ef469eaf11c18e210ec6bc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:57.181474 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:57.183569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:57.219727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:57.268450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d3ff-2c50-40d4-8532-6488db8c301d tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:57.322538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:57.324691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:57.355180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:57.355950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:57.356313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:57.356682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] PATH_INFO: `/v3/users/e6cc144c0d804a84a8d35eaa57fab6bb` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:57.357849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6cc144c0d804a84a8d35eaa57fab6bb)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:57.359891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:57.394167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e13200-2c05-4b1a-a720-92765ac47f26 tempest-VolumeQuotaClassesTest-1429761186 tempest-VolumeQuotaClassesTest-1429761186-project-admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:57.399231 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:57.399900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:57.400179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:57.400449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] PATH_INFO: `/v3/users/4dc16ee508b3449ea9db4883b033045f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:57.401626 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc16ee508b3449ea9db4883b033045f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:57.403649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:57.412512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6cc144c0d804a84a8d35eaa57fab6bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.421465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6cc144c0d804a84a8d35eaa57fab6bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.422172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6cc144c0d804a84a8d35eaa57fab6bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.422786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6cc144c0d804a84a8d35eaa57fab6bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.430753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f114e925-3f42-4c44-acc6-b20c649d0ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6cc144c0d804a84a8d35eaa57fab6bb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.452113 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1300/2598] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:57 2022] DELETE /identity/v3/users/e6cc144c0d804a84a8d35eaa57fab6bb => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:57.455967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc16ee508b3449ea9db4883b033045f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.470701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc16ee508b3449ea9db4883b033045f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.478478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc16ee508b3449ea9db4883b033045f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.479253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc16ee508b3449ea9db4883b033045f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.486610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc16ee508b3449ea9db4883b033045f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.493273 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1299/2599] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:57 2022] DELETE /identity/v3/users/4dc16ee508b3449ea9db4883b033045f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:57.498445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:57.498730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:57.498997 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:57.522705 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:57.527576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:57.595466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:57.615685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1301/2600] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:57.628018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:57.629856 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:57.641882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:57.699219 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:57.700698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:57.720535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9070b-8301-4b83-90a3-31b1830838e3 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:57.727179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:57.727783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:57.728085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:57.728329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] PATH_INFO: `/v3/projects/f5c3250304054594abbaa610e49da216` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:57.729524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c3250304054594abbaa610e49da216)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:57.732693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:57.762544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e655a91-b403-4112-8ebd-a6c842fa7e68 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:57.770255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:57.771243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:57.771558 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:57.771834 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:57.786440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:57.789877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:57.804250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1300/2601] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:57.818919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c3250304054594abbaa610e49da216', 'request_id': 'req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.831134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c3250304054594abbaa610e49da216', 'request_id': 'req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.841680 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c3250304054594abbaa610e49da216', 'request_id': 'req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.856238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] The token cache is being invalidate because project f5c3250304054594abbaa610e49da216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:57.856534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.857260 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:57.859863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1302/2602] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:57 2022] DELETE /identity/v3/projects/f5c3250304054594abbaa610e49da216 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:57.864245 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:57.893666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c8a5d-d3b4-4870-9a6f-6c38678897e7 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:57.961124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:57.962299 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:57.962886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:57.962886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] PATH_INFO: `/v3/users/977642e6e8404054943b619524e5078b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:57.964320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=977642e6e8404054943b619524e5078b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:57.966529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.039010 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '977642e6e8404054943b619524e5078b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.062771 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '977642e6e8404054943b619524e5078b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.092396 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '977642e6e8404054943b619524e5078b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.093321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '977642e6e8404054943b619524e5078b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.118897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '977642e6e8404054943b619524e5078b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.119300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1301/2603] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:57 2022] DELETE /identity/v3/users/977642e6e8404054943b619524e5078b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:58.197106 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.198652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.214090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b742d9-48f5-40cd-a1a8-5aa870d67ea3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.311173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.311862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.312144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.312414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] PATH_INFO: `/v3/projects/acc43c9afbc443e59b8aae4fc491cfae` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.313596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc43c9afbc443e59b8aae4fc491cfae)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:58.315604 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.345276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.347118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.369508 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b7bd2-b854-441f-be69-36d602545408 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.381996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.381996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.381996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.381996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] PATH_INFO: `/v3/projects/916b19fbba7d4e16ac17b6057c07180e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.384224 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=916b19fbba7d4e16ac17b6057c07180e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:58.388294 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.431035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:58.441070 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc43c9afbc443e59b8aae4fc491cfae', 'request_id': 'req-c4b99593-9c48-4e48-b430-4f3c2003300d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.456583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc43c9afbc443e59b8aae4fc491cfae', 'request_id': 'req-c4b99593-9c48-4e48-b430-4f3c2003300d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.470424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc43c9afbc443e59b8aae4fc491cfae', 'request_id': 'req-c4b99593-9c48-4e48-b430-4f3c2003300d'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.484744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] The token cache is being invalidate because project acc43c9afbc443e59b8aae4fc491cfae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:58.485107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.485765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.489275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1303/2604] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:58 2022] DELETE /identity/v3/projects/acc43c9afbc443e59b8aae4fc491cfae => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:58.492450 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.495007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.504201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:58.509843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916b19fbba7d4e16ac17b6057c07180e', 'request_id': 'req-2422eb7a-4bc2-4550-a707-746a328b1912'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.524841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916b19fbba7d4e16ac17b6057c07180e', 'request_id': 'req-2422eb7a-4bc2-4550-a707-746a328b1912'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.542800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916b19fbba7d4e16ac17b6057c07180e', 'request_id': 'req-2422eb7a-4bc2-4550-a707-746a328b1912'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.557281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] The token cache is being invalidate because project 916b19fbba7d4e16ac17b6057c07180e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:58.557570 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.559193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2422eb7a-4bc2-4550-a707-746a328b1912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.561522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1302/2605] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:58 2022] DELETE /identity/v3/projects/916b19fbba7d4e16ac17b6057c07180e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:52:58.566565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.567095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.567371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.589433 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:58.590747 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.595379 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b99593-9c48-4e48-b430-4f3c2003300d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.654363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1303/2606] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:58.656597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.657252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.657905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.658186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.658321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.658523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] PATH_INFO: `/v3/users/c1212f061c49489eb79aa5563fa55f52` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.659727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1212f061c49489eb79aa5563fa55f52)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:58.661738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.671734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc16108-4e27-43a9-bd81-b3f4fa724454 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.676947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.677573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.677848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.678136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] PATH_INFO: `/v3/users/f6d9ed3175fd43b58118174548c0c69c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.683302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d9ed3175fd43b58118174548c0c69c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:58.685307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.718848 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1212f061c49489eb79aa5563fa55f52'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.727167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1212f061c49489eb79aa5563fa55f52'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.727881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1212f061c49489eb79aa5563fa55f52'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.728519 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1212f061c49489eb79aa5563fa55f52'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.736029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d9ed3175fd43b58118174548c0c69c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.737199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-a8f0817f-a23e-489b-b75a-ccd60629f5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1212f061c49489eb79aa5563fa55f52'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.749029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d9ed3175fd43b58118174548c0c69c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.755940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1304/2607] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:58 2022] DELETE /identity/v3/users/c1212f061c49489eb79aa5563fa55f52 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:58.759372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.759372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.759620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d9ed3175fd43b58118174548c0c69c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.759723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.759827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d9ed3175fd43b58118174548c0c69c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.767084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-13948665-4a73-4bd1-a13f-43544dc71c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d9ed3175fd43b58118174548c0c69c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:58.773697 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1304/2608] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:58 2022] DELETE /identity/v3/users/f6d9ed3175fd43b58118174548c0c69c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:58.779285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.779523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.779867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.780873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:58.782165 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.802818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:58.803723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.815345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:58.830924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:58.867045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1305/2609] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:58.872549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.883010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c6792-c92e-49d6-9e04-04de43a7b512 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.889772 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.890402 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.890679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.890948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.898647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:58.898647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:58.898976 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1305/2610] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:58.899104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.906609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.908235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:58.915407 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1306/2611] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:58.922299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:58.937079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516c82-3a52-40be-ab6e-b6a2581d6f29 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.955881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.956512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.956799 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.957089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.957758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:58.960424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:58.988129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32002972-9501-43ce-9030-49ba0b4ca832 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:58.992324 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:58.993025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:58.993186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1307/2612] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:58 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:58.993549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:58.993963 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:58.998053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.004957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.006502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.017372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1306/2613] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:52:59.080064 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29a093-0c66-4eab-b9c8-d5e595ddcacb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.085600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.086327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.086729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.087073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.087835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.089968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.096175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.097600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.125424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1308/2614] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:58 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:52:59.126853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.137685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317a14c-874f-47f9-b74e-743a89bbc94b service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.139472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72bc9-7bbe-49cf-ad25-599525cc81a9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.144216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.144861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.145001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.145368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.145687 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.145835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] PATH_INFO: `/v3/projects/224952a5828b412291619f5f8cbd795f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.146100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.146446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.147103 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224952a5828b412291619f5f8cbd795f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.147743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.149501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.149896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.163698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1309/2615] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:59.169092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.183682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a38ede-4e3d-448a-98d1-d5f2b1cf8d0f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.190198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.191226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.191565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.191801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.192483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.195388 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.213505 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1310/2616] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:52:59.214589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.226476 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757a204-3a37-4bb6-a9ec-990cd777aee0 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.228370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224952a5828b412291619f5f8cbd795f', 'request_id': 'req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.240362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224952a5828b412291619f5f8cbd795f', 'request_id': 'req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.251892 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224952a5828b412291619f5f8cbd795f', 'request_id': 'req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.275543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] The token cache is being invalidate because project 224952a5828b412291619f5f8cbd795f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:59.275931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.277091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dbe9b0b3-38b6-4892-aaf4-f10a142f0882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.282741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1307/2617] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:59 2022] DELETE /identity/v3/projects/224952a5828b412291619f5f8cbd795f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:59.284839 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afe60-c20a-4f16-9a39-209baf5e63a3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.285323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afe60-c20a-4f16-9a39-209baf5e63a3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.285690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afe60-c20a-4f16-9a39-209baf5e63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.313763 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bc6afe60-c20a-4f16-9a39-209baf5e63a3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:59.315085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bc6afe60-c20a-4f16-9a39-209baf5e63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.359466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.360334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.360971 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.362833 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] PATH_INFO: `/v3/projects/d308ae5684724251aa1011f269cf466a/users/94533495d9ea4e489e7283dbf307ae66/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.373906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:59.374298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:59.384599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d308ae5684724251aa1011f269cf466a, user_id=94533495d9ea4e489e7283dbf307ae66, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.388767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.398791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1308/2618] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:59.407277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e34af-c454-4979-8532-435998bd18d9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.407595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e34af-c454-4979-8532-435998bd18d9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.407867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e34af-c454-4979-8532-435998bd18d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.413374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1311/2619] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:52:59 2022] PUT /identity/v3/projects/d308ae5684724251aa1011f269cf466a/users/94533495d9ea4e489e7283dbf307ae66/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:59.414220 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.417765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.431546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b59e34af-c454-4979-8532-435998bd18d9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:59.433850 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b59e34af-c454-4979-8532-435998bd18d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.489581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5fe60-1d79-4b50-a026-cb33883f58c5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.494879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.495744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.496103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.496443 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] PATH_INFO: `/v3/projects/bf898e26fa1241ba8e236f4c5069fce0` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.497678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf898e26fa1241ba8e236f4c5069fce0)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.499717 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.506013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1309/2620] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:59.511202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.511820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.512539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.558545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf898e26fa1241ba8e236f4c5069fce0', 'request_id': 'req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.567897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf898e26fa1241ba8e236f4c5069fce0', 'request_id': 'req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.576351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf898e26fa1241ba8e236f4c5069fce0', 'request_id': 'req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.589668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] The token cache is being invalidate because project bf898e26fa1241ba8e236f4c5069fce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:52:59.590057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.591015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.592879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1312/2621] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:52:59 2022] DELETE /identity/v3/projects/bf898e26fa1241ba8e236f4c5069fce0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:59.593483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.594880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.661150 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:52:59.662664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.674120 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1310/2622] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:52:59.675051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.676634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.684567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836fdf8-b8ae-4fbc-a3a0-b17a23be56b7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.690006 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.690292 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2805a66-d002-4cca-aeae-e5534b5622ea None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.690629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.690937 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.691166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.691870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:52:59.692173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.694511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.695859 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.696517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.696802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.697147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] PATH_INFO: `/v3/users/737d46f71984476aa3e0e801b22ad6a8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.698651 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=737d46f71984476aa3e0e801b22ad6a8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.700841 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.709028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1313/2623] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:52:59.709883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.713126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.727092 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:59.764269 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737d46f71984476aa3e0e801b22ad6a8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.787668 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737d46f71984476aa3e0e801b22ad6a8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.788559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82f1de-808d-42d6-846e-d4d298e99cfc admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.793569 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.794303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.794863 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.795269 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.796861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737d46f71984476aa3e0e801b22ad6a8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.797853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737d46f71984476aa3e0e801b22ad6a8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.806183 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.806367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737d46f71984476aa3e0e801b22ad6a8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:52:59.807856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.813313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1311/2624] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:52:59 2022] DELETE /identity/v3/users/737d46f71984476aa3e0e801b22ad6a8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:52:59.814407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.815995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:52:59.818259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1314/2625] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:52:59.886969 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:52:59.892965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c128db8-0cba-4335-8a34-50ba1432c11b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:52:59.899588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:52:59.900216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:52:59.900742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:52:59.901047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:52:59.901738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:52:59.903823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:52:59.935249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1312/2626] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:52:59 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:52:59.939414 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:52:59.949984 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3b72-ded9-46ac-9a61-ad7e9e64455e service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.028320 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.029048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.029411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.030091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.030556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.032796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.060890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1315/2627] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:52:59 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:00.065648 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.082532 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e0e36-34a4-4541-86d7-215b75017f50 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.106242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.107260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.107443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.107936 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.112380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.115255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.133228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1313/2628] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:53:00.137653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.151044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679056f3-0fdd-4737-9338-7d89c1f3b754 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.157366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa767a6-033a-4e53-8414-8835b871feea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.158153 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa767a6-033a-4e53-8414-8835b871feea admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.158533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa767a6-033a-4e53-8414-8835b871feea admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.158921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa767a6-033a-4e53-8414-8835b871feea admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.159828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa767a6-033a-4e53-8414-8835b871feea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.161654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa767a6-033a-4e53-8414-8835b871feea admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.181769 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1316/2629] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:00.186222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.196990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4501c-4492-49df-acfc-1f62ecb1c1b3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.202828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.203553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.203836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.204679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] PATH_INFO: `/v3/projects/abad6fe318944efa9510215cf23c0f43/users/c37e4673794648b581eaeca98e3e8fff/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.223044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abad6fe318944efa9510215cf23c0f43, user_id=c37e4673794648b581eaeca98e3e8fff, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.225992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.247115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1314/2630] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:00 2022] PUT /identity/v3/projects/abad6fe318944efa9510215cf23c0f43/users/c37e4673794648b581eaeca98e3e8fff/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:00.257509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.259368 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.259847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.312446 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] MFA Rules not processed for user `c37e4673794648b581eaeca98e3e8fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:00.315549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:00.422480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1317/2631] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:00.427667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.463083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.468104 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:00.531519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d50dd1-4a27-4564-8758-910a07b74b25 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.538814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.539679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.540096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.540650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.542782 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.544066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:00.548947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.598503 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1315/2632] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:00 2022] POST /identity/v3/projects => generated 389 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:00.604406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.616049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62df33c-375c-48aa-a12f-357a4fae9507 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.710118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132fbd-e18e-4d38-b56b-471ed8db7bd4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.718418 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.719024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.719309 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.719751 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.720453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.722804 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.773178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1316/2633] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:00 2022] POST /identity/v3/users => generated 301 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:00.774923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.789097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.789097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.796392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.796874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] PATH_INFO: `/v3/projects/16c696a6c9244241967a75eca14b10e5` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.799650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c696a6c9244241967a75eca14b10e5)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.803564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.804053 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5dd048-45d9-46c6-a4c8-269a543dfbda admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.823256 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.826502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.826798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.827077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.827829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.831789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.858951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1317/2634] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:00.866427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.883411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f9e68-8bd6-41cd-a112-0dd23492e1a7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:00.896077 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:00.896807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:00.897094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:00.897394 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] PATH_INFO: `/v3/projects/af4ef8ba09e04ecc9050595a735791da/users/dbc163fae72c434da55db7b93b21f0c2/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:00.906347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:00.914464 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c696a6c9244241967a75eca14b10e5', 'request_id': 'req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:00.932102 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c696a6c9244241967a75eca14b10e5', 'request_id': 'req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:00.935993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4ef8ba09e04ecc9050595a735791da, user_id=dbc163fae72c434da55db7b93b21f0c2, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:00.938739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:00.943847 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c696a6c9244241967a75eca14b10e5', 'request_id': 'req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:00.969729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1318/2635] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:00 2022] PUT /identity/v3/projects/af4ef8ba09e04ecc9050595a735791da/users/dbc163fae72c434da55db7b93b21f0c2/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:00.973710 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.976564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] The token cache is being invalidate because project 16c696a6c9244241967a75eca14b10e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:00.977137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:00.977887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:00.980180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1318/2636] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:00 2022] DELETE /identity/v3/projects/16c696a6c9244241967a75eca14b10e5 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:00.988099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:00.990035 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.073002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e747e92-03c9-43f3-a745-e01e68ae1e4f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.080951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0eb6-3dcd-412f-9b64-dc2699b02527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.081613 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0eb6-3dcd-412f-9b64-dc2699b02527 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.081957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0eb6-3dcd-412f-9b64-dc2699b02527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.082253 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0eb6-3dcd-412f-9b64-dc2699b02527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.082990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0eb6-3dcd-412f-9b64-dc2699b02527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.084720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0eb6-3dcd-412f-9b64-dc2699b02527 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.102138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1319/2637] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:53:01.103340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.103689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.103949 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.128211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:01.129509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.135099 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:01.139332 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1320/2638] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:01.140200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.145444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223a5f9-e8a8-42ba-ba1a-ff09d6744319 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.147102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.163828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86f9bb-562c-4b70-9c26-259f5237fcdd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.176200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f13b2-037c-4fed-abd3-d64a5f1aeca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.177537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f13b2-037c-4fed-abd3-d64a5f1aeca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.177537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f13b2-037c-4fed-abd3-d64a5f1aeca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.177537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f13b2-037c-4fed-abd3-d64a5f1aeca2 admin admin] PATH_INFO: `/v3/projects/af4ef8ba09e04ecc9050595a735791da/users/dbc163fae72c434da55db7b93b21f0c2/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.184109 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f13b2-037c-4fed-abd3-d64a5f1aeca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4ef8ba09e04ecc9050595a735791da, user_id=dbc163fae72c434da55db7b93b21f0c2, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.188430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f13b2-037c-4fed-abd3-d64a5f1aeca2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.211783 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1321/2639] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:01 2022] PUT /identity/v3/projects/af4ef8ba09e04ecc9050595a735791da/users/dbc163fae72c434da55db7b93b21f0c2/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:01.213307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.213592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.214005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.238034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:01.239326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.252449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.253082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.253363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.253659 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.256618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.272995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.274746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf102f-acbf-4401-ba98-ba51b2c791ee service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.288702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1319/2640] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:01.289961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.290293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.290653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.325313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] MFA Rules not processed for user `dbc163fae72c434da55db7b93b21f0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:01.327236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.361725 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1322/2641] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:01.374742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.397327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64763-90c4-4611-8730-9b853ae177b5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.405635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.406281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.406563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.406828 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.407573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:01.407872 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.409756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.423458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1323/2642] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:53:01.428059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.443609 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e9c5-92fc-41f4-b228-05a94be077b8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.450254 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.451084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.451468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.451812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.452535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.455025 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.456812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1320/2643] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:01.472230 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.474060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.508951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1324/2644] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:01 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:01.513549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.556013 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:01.610343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763934d5-bc4e-42ee-b752-d453f7f9e500 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.623467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.624441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.624878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.625317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.627141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.631816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.687011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1325/2645] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:01 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:01.688091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.715214 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563bf40-ea11-4173-8e58-16d28c79115d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.734413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.735034 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.735315 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.737960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.737960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.739355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.752829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01596a-5b62-4bfb-8952-66dd9ab73ad0 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.763384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1326/2646] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:01.767743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.780464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c1e56-468a-460d-bfd9-bea17bb30675 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.788549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.789165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.789441 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.790513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.791184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.793855 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.800024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:01.810743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1327/2647] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:53:01.814760 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:01.834959 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.835968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.836236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.836501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.851070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6850c-043d-402e-9073-6f839e2d3a5a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:01.859847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65fb38-94fc-449a-8b0f-42dd871f5066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:01.860557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65fb38-94fc-449a-8b0f-42dd871f5066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.860835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65fb38-94fc-449a-8b0f-42dd871f5066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.861107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65fb38-94fc-449a-8b0f-42dd871f5066 admin admin] PATH_INFO: `/v3/projects/7190d1f280a5412da6ecf00903c60464/users/5d1ef08a076f4c8e8debd180dafd6ca5/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.888093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65fb38-94fc-449a-8b0f-42dd871f5066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7190d1f280a5412da6ecf00903c60464, user_id=5d1ef08a076f4c8e8debd180dafd6ca5, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.891234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65fb38-94fc-449a-8b0f-42dd871f5066 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.918110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1328/2648] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:01 2022] PUT /identity/v3/projects/7190d1f280a5412da6ecf00903c60464/users/5d1ef08a076f4c8e8debd180dafd6ca5/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:01.931644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:01.932211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:01.933393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:01.973121 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:01.975222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:01.977310 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] MFA Rules not processed for user `5d1ef08a076f4c8e8debd180dafd6ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:01.979187 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:01.989028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1321/2649] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:02.100923 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1329/2650] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:02.114136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.134284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ad397-08db-43ab-9e22-0998d7c2c0e3 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.214134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.292534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.295068 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.295662 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.296072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.297014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.299356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.347501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1322/2651] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:02 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:02.351767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.384920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a72950-e414-43d1-a89c-86c785790e71 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.393852 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:02.463075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.463745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.464045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.464319 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] PATH_INFO: `/v3/users/c37e4673794648b581eaeca98e3e8fff` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.465527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37e4673794648b581eaeca98e3e8fff)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.467528 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.474175 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f4dd6-f66e-4037-8b44-d4eaacbdda02 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.479642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.480295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.480589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.480896 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.481634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.484442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.503950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:02.513454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1323/2652] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:02 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:02.517091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.530110 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37e4673794648b581eaeca98e3e8fff'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:02.530674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fbd6b-8c06-4798-9c38-92cf251cbb82 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.536581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.537621 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.537981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.538257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.538914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.540602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.546152 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37e4673794648b581eaeca98e3e8fff'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:02.555647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1324/2653] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:02.560111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37e4673794648b581eaeca98e3e8fff'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:02.561050 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37e4673794648b581eaeca98e3e8fff'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:02.561174 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.571007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-edfa1684-dd8d-40fd-a312-3b965b06ee1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37e4673794648b581eaeca98e3e8fff'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:02.577304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fab38-2c64-491b-88ba-ac4b97786e66 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.580186 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1330/2654] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:02 2022] DELETE /identity/v3/users/c37e4673794648b581eaeca98e3e8fff => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:02.584621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.585472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.585472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.586970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.587774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.588116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.588453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] PATH_INFO: `/v3/projects/76e1406afaf84370a5042fdf55b3cc85/users/afeff130542f40c2bc71dd26bf9322c6/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.605711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e1406afaf84370a5042fdf55b3cc85, user_id=afeff130542f40c2bc71dd26bf9322c6, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.608513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.610880 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:02.612161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:02.627786 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1325/2655] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:02 2022] PUT /identity/v3/projects/76e1406afaf84370a5042fdf55b3cc85/users/afeff130542f40c2bc71dd26bf9322c6/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:02.631910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.634988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:02.686631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1331/2656] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:02.694046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.695670 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:02.726401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745327d-3322-42f4-9838-62c91cc8599c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.734495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.735703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:02.736441 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.736574 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.736828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.738600 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.741836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.755619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1326/2657] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:02.760151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.775739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1833f-9615-4925-8e31-26a01680abe9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.777861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee21b68-fc2b-4ad7-bbef-71c28298afb3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:02.786606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d881b-c195-4695-bdc7-fd7d583f63c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.787311 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d881b-c195-4695-bdc7-fd7d583f63c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.787589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d881b-c195-4695-bdc7-fd7d583f63c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.787860 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d881b-c195-4695-bdc7-fd7d583f63c7 admin admin] PATH_INFO: `/v3/projects/76e1406afaf84370a5042fdf55b3cc85/users/afeff130542f40c2bc71dd26bf9322c6/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.789890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d881b-c195-4695-bdc7-fd7d583f63c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e1406afaf84370a5042fdf55b3cc85, user_id=afeff130542f40c2bc71dd26bf9322c6, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.793310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d881b-c195-4695-bdc7-fd7d583f63c7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.807226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1327/2658] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:02 2022] PUT /identity/v3/projects/76e1406afaf84370a5042fdf55b3cc85/users/afeff130542f40c2bc71dd26bf9322c6/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:02.819948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.820232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.820504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.842290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:02.843221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:02.844704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:02.845055 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:02.852477 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] MFA Rules not processed for user `afeff130542f40c2bc71dd26bf9322c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:02.854048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:02.859381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:02.861527 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:02.894998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1332/2659] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:53:02.977949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1328/2660] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:02.986007 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:02.987727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:03.166800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f9e2-93a9-4f9c-8ee3-0f1151afeda6 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:03.274002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:03.274727 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:03.275071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:03.275449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:03.288141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:03.289947 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:03.312679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1333/2661] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:04.173323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:04.175516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:04.245044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:04.288692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad7980-f738-48fd-acbc-226381435e18 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:04.296159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:04.297047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:04.297417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:04.298423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] PATH_INFO: `/v3/projects/abad6fe318944efa9510215cf23c0f43` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:04.299920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abad6fe318944efa9510215cf23c0f43)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:04.302047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:04.385516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abad6fe318944efa9510215cf23c0f43', 'request_id': 'req-bbe2d848-36b3-4508-a64a-21d3552db013'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.396952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abad6fe318944efa9510215cf23c0f43', 'request_id': 'req-bbe2d848-36b3-4508-a64a-21d3552db013'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.404747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abad6fe318944efa9510215cf23c0f43', 'request_id': 'req-bbe2d848-36b3-4508-a64a-21d3552db013'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.418748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] The token cache is being invalidate because project abad6fe318944efa9510215cf23c0f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:04.419050 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.420071 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.425756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1329/2662] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:04 2022] DELETE /identity/v3/projects/abad6fe318944efa9510215cf23c0f43 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:04.427649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:04.445535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7d148-de8f-49b8-9911-cebd90f7bd2b service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:04.508701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:04.510524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:04.511971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:04.548227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:04.548227 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:04.548586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:04.548866 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] PATH_INFO: `/v3/users/dbc163fae72c434da55db7b93b21f0c2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:04.551092 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc163fae72c434da55db7b93b21f0c2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:04.553111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:04.628394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d848-36b3-4508-a64a-21d3552db013 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:04.635491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:04.638082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:04.638490 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:04.638793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] PATH_INFO: `/v3/users/5d1ef08a076f4c8e8debd180dafd6ca5` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:04.640725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1ef08a076f4c8e8debd180dafd6ca5)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:04.643983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:04.649958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc163fae72c434da55db7b93b21f0c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.688306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc163fae72c434da55db7b93b21f0c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.703054 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc163fae72c434da55db7b93b21f0c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.704205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc163fae72c434da55db7b93b21f0c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.716547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-87076eed-62d3-4999-be8c-ba8051efef4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc163fae72c434da55db7b93b21f0c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.725604 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1334/2663] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:04 2022] DELETE /identity/v3/users/dbc163fae72c434da55db7b93b21f0c2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:04.737619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1ef08a076f4c8e8debd180dafd6ca5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.750934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1ef08a076f4c8e8debd180dafd6ca5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.751911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1ef08a076f4c8e8debd180dafd6ca5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.752939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1ef08a076f4c8e8debd180dafd6ca5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.775334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1ef08a076f4c8e8debd180dafd6ca5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:04.778367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:04.822722 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1330/2664] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:04 2022] DELETE /identity/v3/users/5d1ef08a076f4c8e8debd180dafd6ca5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:04.828769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:04.829440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:04.829800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:04.871897 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:04.873539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:05.065636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1335/2665] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:05.083471 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:05.085181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:05.216795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635282e-9fe9-4b9b-b2f0-a5c9310bfcf8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:05.222437 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:05.223383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:05.224117 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:05.224542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:05.240289 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:05.242741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:05.256047 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1331/2666] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:05.546381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:05.546381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:05.565636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91688fb9-d5bf-4b55-b923-9d87bae21b76 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:05.592935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:05.593593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:05.593867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:05.594133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] PATH_INFO: `/v3/projects/af4ef8ba09e04ecc9050595a735791da` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:05.595370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4ef8ba09e04ecc9050595a735791da)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:05.597376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:05.701562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4ef8ba09e04ecc9050595a735791da', 'request_id': 'req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:05.721470 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4ef8ba09e04ecc9050595a735791da', 'request_id': 'req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:05.725130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:05.731546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4ef8ba09e04ecc9050595a735791da', 'request_id': 'req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:05.740771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:05.742770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:05.764925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b719e1-69f7-4ce3-961d-44b8a8233722 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:05.771691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] The token cache is being invalidate because project af4ef8ba09e04ecc9050595a735791da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:05.775141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:05.780087 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-dfda3fc8-dd17-4f59-903a-1fe9d2bbacbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:05.783462 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1336/2667] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:05 2022] DELETE /identity/v3/projects/af4ef8ba09e04ecc9050595a735791da => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:05.789184 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dfbdc-23f5-42e1-af71-381edb432213 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:05.789568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dfbdc-23f5-42e1-af71-381edb432213 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:05.789906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dfbdc-23f5-42e1-af71-381edb432213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:05.829138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-755dfbdc-23f5-42e1-af71-381edb432213 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:05.830576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-755dfbdc-23f5-42e1-af71-381edb432213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:05.931211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1337/2668] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:05.937876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:05.938327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:05.938704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:05.946616 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:05.947276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:05.947581 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:05.947852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] PATH_INFO: `/v3/projects/7190d1f280a5412da6ecf00903c60464` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:05.951357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7190d1f280a5412da6ecf00903c60464)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:05.962893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:05.964997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:05.968118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:05.974593 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1338/2669] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:05.978979 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:05.990354 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e2ca-0252-4856-9a90-e2bddbd80d48 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:05.996388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:05.997111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:05.997544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:05.998382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:05.998737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:05.999141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.000990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.021513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1339/2670] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:53:06.023204 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.038076 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631d62-e336-4659-8ab7-60d0271e62a8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.060929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7190d1f280a5412da6ecf00903c60464', 'request_id': 'req-4f035c99-602b-4725-a91b-d82707fd68ac'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.091551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7190d1f280a5412da6ecf00903c60464', 'request_id': 'req-4f035c99-602b-4725-a91b-d82707fd68ac'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.107623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7190d1f280a5412da6ecf00903c60464', 'request_id': 'req-4f035c99-602b-4725-a91b-d82707fd68ac'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.111274 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:06.115891 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.116443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.116728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.117288 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.118513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.122113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.149237 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] The token cache is being invalidate because project 7190d1f280a5412da6ecf00903c60464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:06.149691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.150467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.155603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1332/2671] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:05 2022] DELETE /identity/v3/projects/7190d1f280a5412da6ecf00903c60464 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:06.157067 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.159037 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.180023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1340/2672] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:06 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:06.181584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.183603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035c99-602b-4725-a91b-d82707fd68ac admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.187862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.307466 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:06.325293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a410d-15d7-4311-a129-04defa8a55be admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.333631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68434f42-bd7d-4028-abbc-dd7ef8f73730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.335022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434f42-bd7d-4028-abbc-dd7ef8f73730 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.335452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434f42-bd7d-4028-abbc-dd7ef8f73730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.335796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434f42-bd7d-4028-abbc-dd7ef8f73730 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.336851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68434f42-bd7d-4028-abbc-dd7ef8f73730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.340260 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68434f42-bd7d-4028-abbc-dd7ef8f73730 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.352920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.354177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.355098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.356511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] PATH_INFO: `/v3/users/afeff130542f40c2bc71dd26bf9322c6` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.359357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afeff130542f40c2bc71dd26bf9322c6)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.362826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.383160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1341/2673] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:06 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:06.385234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.386756 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.387357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.422216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] MFA Rules not processed for user `94533495d9ea4e489e7283dbf307ae66`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:06.424061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.447548 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afeff130542f40c2bc71dd26bf9322c6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.457509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afeff130542f40c2bc71dd26bf9322c6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.458533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afeff130542f40c2bc71dd26bf9322c6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.460353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afeff130542f40c2bc71dd26bf9322c6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.472727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afeff130542f40c2bc71dd26bf9322c6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:06.499491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1333/2674] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:06 2022] DELETE /identity/v3/users/afeff130542f40c2bc71dd26bf9322c6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:06.499491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.500373 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.559359 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1342/2675] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:06.560090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.561772 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.579300 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ec5e0-8060-4fa1-966e-02f1505d3c29 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.624725 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43163e-603a-40e8-be7f-db440156dd59 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.631455 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.631754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.632031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.632351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.632942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.634756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.647642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1334/2676] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:06.652251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.653989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.654738 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.654896 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.655234 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.655921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.658095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.662515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0c27a-6619-4b57-a0fa-1bffbbd50d44 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.667656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.668326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.668586 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.669265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.669906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.671828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.674280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:06.689567 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1335/2677] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:06.690683 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.700845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1343/2678] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:06 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:06.705443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.708214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b478166-fb8d-4996-9318-7ec7f0e41046 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.793917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d34c9-551f-4901-a7c5-00d952187de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.794949 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d34c9-551f-4901-a7c5-00d952187de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.795280 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d34c9-551f-4901-a7c5-00d952187de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.795591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d34c9-551f-4901-a7c5-00d952187de6 admin admin] PATH_INFO: `/v3/projects/1257196b4115437cb5822efb45e12c59/users/4e7b92a78a0e4389a7fe6e54e1e33a25/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.800785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:06.805116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818267b8-3e15-4791-8324-bf837a598ffc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.813427 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.814864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.815362 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.815957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.816649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.818596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d34c9-551f-4901-a7c5-00d952187de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1257196b4115437cb5822efb45e12c59, user_id=4e7b92a78a0e4389a7fe6e54e1e33a25, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.819712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.821449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d34c9-551f-4901-a7c5-00d952187de6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.839897 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1336/2679] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:06 2022] PUT /identity/v3/projects/1257196b4115437cb5822efb45e12c59/users/4e7b92a78a0e4389a7fe6e54e1e33a25/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:06.845467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.845827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.846138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.847381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1344/2680] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:06 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:06.851912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.868251 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:06.869660 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.872550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf569508-33d7-4c7e-b7fc-d24184d815db admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.953625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1337/2681] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:06.953873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.954511 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.954858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.955033 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.955248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.956017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.957778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:06.964254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.970786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1345/2682] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:53:06.971791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:06.973806 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:06.985900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5ad7-3337-4e67-a8e5-ddebbbb4628e None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:06.992126 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:06.992575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:06.992874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:06.993129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] PATH_INFO: `/v3/projects/76e1406afaf84370a5042fdf55b3cc85` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:06.994333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e1406afaf84370a5042fdf55b3cc85)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:06.996888 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.053117 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4efe5-52a0-4856-b3d7-877db135843c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.061222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e1406afaf84370a5042fdf55b3cc85', 'request_id': 'req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:07.073679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e1406afaf84370a5042fdf55b3cc85', 'request_id': 'req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:07.085432 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e1406afaf84370a5042fdf55b3cc85', 'request_id': 'req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:07.115473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] The token cache is being invalidate because project 76e1406afaf84370a5042fdf55b3cc85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:07.115473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:07.116203 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:07.117908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1338/2683] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:06 2022] DELETE /identity/v3/projects/76e1406afaf84370a5042fdf55b3cc85 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:07.118741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.120924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:07.128041 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:07.138838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.139758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.140037 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.140301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.142737 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:07.209025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbbcfd-e4e2-41f7-82ea-892ed8ae664a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.214513 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.216083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.216360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.216626 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] PATH_INFO: `/v3/projects/f87fa1a153374028977e76e49f5626bb/users/b1e8669f366f47e0938ed3a4a1024081/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.218808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:07.232958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87fa1a153374028977e76e49f5626bb, user_id=b1e8669f366f47e0938ed3a4a1024081, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.234064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.235677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.235826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb8ac1-4a83-4644-a868-5873dc73b9c3 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.248510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1346/2684] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:07.249768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb58483-ee4a-4bda-9cd5-f906c50a8dbd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.250059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb58483-ee4a-4bda-9cd5-f906c50a8dbd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.250320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb58483-ee4a-4bda-9cd5-f906c50a8dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.256351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1339/2685] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:07 2022] PUT /identity/v3/projects/f87fa1a153374028977e76e49f5626bb/users/b1e8669f366f47e0938ed3a4a1024081/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:07.257257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.275836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-adb58483-ee4a-4bda-9cd5-f906c50a8dbd None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:07.278543 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-adb58483-ee4a-4bda-9cd5-f906c50a8dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:07.334424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e5a4-247f-4ee0-801c-f6f60ffecb7f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.340564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.341276 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.341647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.342042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.342693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.344601 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.360158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1340/2686] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:07.360821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1347/2687] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:07.361534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.367793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.367909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.368116 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.375177 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3721109-5aef-4c2e-9c75-025bc9fca312 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.380839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.381641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.381988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.382319 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] PATH_INFO: `/v3/projects/f87fa1a153374028977e76e49f5626bb/users/b1e8669f366f47e0938ed3a4a1024081/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.389939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87fa1a153374028977e76e49f5626bb, user_id=b1e8669f366f47e0938ed3a4a1024081, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.391954 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:07.393171 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:07.393303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.401704 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1348/2688] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:07.406928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.410932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1341/2689] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:07 2022] PUT /identity/v3/projects/f87fa1a153374028977e76e49f5626bb/users/b1e8669f366f47e0938ed3a4a1024081/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:07.421967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17f840-d3ca-4058-ae76-ba23d490087f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.543883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.546127 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.547718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.548016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.551501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:07.552021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.558799 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.575732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1349/2690] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:53:07.581672 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.596957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a594863-3d41-4395-a625-bc824a883980 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.612701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.613504 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.620665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.622815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.624338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.629396 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.693141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1342/2691] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:07 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:07.697244 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.709134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05237326-1027-434c-bc8a-5a0fb3e23bf1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.801403 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.802413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.802884 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.803578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.805167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.807617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.845748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.846752 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1350/2692] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:07 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:07.862214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb4953-6fe2-40cc-b774-db8e7a938455 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.873195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.875235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.876142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.876889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.878008 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.880400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.896960 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1343/2693] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:07.902453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.917810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c479e-c377-4dd3-8993-7b579fe796ea admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.928823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dabcee-7a22-46be-915d-2f8f8536d502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.930068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dabcee-7a22-46be-915d-2f8f8536d502 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.931679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dabcee-7a22-46be-915d-2f8f8536d502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.932130 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dabcee-7a22-46be-915d-2f8f8536d502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:07.933001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dabcee-7a22-46be-915d-2f8f8536d502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:07.935338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dabcee-7a22-46be-915d-2f8f8536d502 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:07.937283 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:07.961917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1351/2694] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:07.963180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:07.975606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea68025-c092-4907-a85d-a96eca714e64 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:07.985060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ae63b-a7e7-437c-9785-7674b9133781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:07.985699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ae63b-a7e7-437c-9785-7674b9133781 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:07.986122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ae63b-a7e7-437c-9785-7674b9133781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:07.986263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ae63b-a7e7-437c-9785-7674b9133781 admin admin] PATH_INFO: `/v3/projects/285c5c85192747cdb72dd884d474c320/users/41ee5ebabbac4138bff8bb1728256e02/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.014598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ae63b-a7e7-437c-9785-7674b9133781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285c5c85192747cdb72dd884d474c320, user_id=41ee5ebabbac4138bff8bb1728256e02, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.020981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ae63b-a7e7-437c-9785-7674b9133781 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.058436 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1344/2695] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:07 2022] PUT /identity/v3/projects/285c5c85192747cdb72dd884d474c320/users/41ee5ebabbac4138bff8bb1728256e02/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:08.061768 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.062250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.062703 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.117903 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] MFA Rules not processed for user `41ee5ebabbac4138bff8bb1728256e02`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:08.121078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:08.139533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.139903 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.140214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.192835 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:08.196740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] MFA Rules not processed for user `4e7b92a78a0e4389a7fe6e54e1e33a25`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:08.198839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:08.233154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1352/2696] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:08.234652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.308015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1345/2697] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:08.311303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.316613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:08.390279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9a45a-9809-49f0-ae22-03aa282c9016 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.395320 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.395975 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.396249 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.397238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.397894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.400017 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.441680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1353/2698] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:08 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:08.445964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.470874 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ec19-b18c-425f-9667-f7c4da75c2a1 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.571129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.571330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8449e-0908-4cdc-a7ed-9d2e4d15b5da admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.572347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.572584 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.572933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.573617 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.578021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.579910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.580535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.581306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.581554 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.582285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.587323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.611023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:08.623066 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1354/2699] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:08 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:08.624729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.627974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1346/2700] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:08 2022] POST /identity/v3/projects => generated 371 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:08.631655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.654657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f116e0-c2ed-4e5a-bf53-68a045639bb0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.689728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:08.720231 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b2f56-473a-42c1-a6c7-0df54bae9454 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.726140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.727116 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.727518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.727933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.728678 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.731558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.783406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1347/2701] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:08 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:08.787049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.791089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.792993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.793664 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.794191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.795386 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.797754 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.808957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf2fbb-30eb-4d46-a641-939cd0627689 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.817481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1355/2702] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:08.822487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.828702 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.829338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.829614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.829878 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.830894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.832305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.844313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ba3d0-b37e-44c1-8557-8a37bdaeb635 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.855147 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1348/2703] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:08.856341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.857176 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.857514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.857736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] PATH_INFO: `/v3/projects/c6110c04a67b4c26835160b622aede96/users/cef6f885f5b34a2898746e4243e78b0b/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.859844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.882491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bf8d6-14df-4365-b3c0-aca8c5c5f75f admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:08.890059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6110c04a67b4c26835160b622aede96, user_id=cef6f885f5b34a2898746e4243e78b0b, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.895308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:08.897226 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.899540 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:08.899882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:08.900115 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] PATH_INFO: `/v3/projects/4eec172a045a4686b7d633cffca2965e/users/801928d69b9a4c0688ed637ec6fa149c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:08.917410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1356/2704] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:08 2022] PUT /identity/v3/projects/c6110c04a67b4c26835160b622aede96/users/cef6f885f5b34a2898746e4243e78b0b/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:08.925729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eec172a045a4686b7d633cffca2965e, user_id=801928d69b9a4c0688ed637ec6fa149c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:08.929270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:08.932692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:08.947836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1349/2705] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:08 2022] PUT /identity/v3/projects/4eec172a045a4686b7d633cffca2965e/users/801928d69b9a4c0688ed637ec6fa149c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:08.952871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:09.023589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2856a3-99f6-4515-b8e2-ede584b6132a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:09.042496 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efb058-779b-4ac7-9778-8c69b2e9cfcc admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:09.049876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:09.050864 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:09.051367 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:09.052058 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:09.053007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:09.056708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:09.079227 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1350/2706] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:09.084463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:09.096906 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5145f-6b43-457e-8cbc-c1866f81ff90 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:09.102322 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:09.103055 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:09.103378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:09.103701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] PATH_INFO: `/v3/projects/4eec172a045a4686b7d633cffca2965e/users/801928d69b9a4c0688ed637ec6fa149c/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:09.106917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eec172a045a4686b7d633cffca2965e, user_id=801928d69b9a4c0688ed637ec6fa149c, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:09.110005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:09.120881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483449c-cd43-468b-92a6-77cb89b53b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:09.121500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483449c-cd43-468b-92a6-77cb89b53b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:09.121926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483449c-cd43-468b-92a6-77cb89b53b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:09.122196 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483449c-cd43-468b-92a6-77cb89b53b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:09.123729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483449c-cd43-468b-92a6-77cb89b53b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:09.126495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1351/2707] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:09 2022] PUT /identity/v3/projects/4eec172a045a4686b7d633cffca2965e/users/801928d69b9a4c0688ed637ec6fa149c/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:09.127655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483449c-cd43-468b-92a6-77cb89b53b08 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:09.140071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1357/2708] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:09.144553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:09.226206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3d0d5-0a91-4f34-965d-4445ca393e52 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:09.231804 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:09.232686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:09.232969 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:09.233246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] PATH_INFO: `/v3/projects/c6110c04a67b4c26835160b622aede96/users/cef6f885f5b34a2898746e4243e78b0b/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:09.235935 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6110c04a67b4c26835160b622aede96, user_id=cef6f885f5b34a2898746e4243e78b0b, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:09.238824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:09.256051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1352/2709] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:09 2022] PUT /identity/v3/projects/c6110c04a67b4c26835160b622aede96/users/cef6f885f5b34a2898746e4243e78b0b/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:09.267916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73413d2-5efb-4c78-8243-0fa1394e7da0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:09.268170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73413d2-5efb-4c78-8243-0fa1394e7da0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:09.268467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73413d2-5efb-4c78-8243-0fa1394e7da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:09.311556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a73413d2-5efb-4c78-8243-0fa1394e7da0 None None] MFA Rules not processed for user `cef6f885f5b34a2898746e4243e78b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:09.313336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a73413d2-5efb-4c78-8243-0fa1394e7da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:09.320015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:09.446829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1358/2710] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:09.463155 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:09.465390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:09.479023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:09.581881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433647b1-562b-430c-bfcb-ea96e2bc5ab5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:09.588636 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:09.590418 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:09.590552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:09.590802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:09.606714 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:09.611466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:09.637286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1353/2711] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:10.217736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:10.218250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:10.218250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:10.220666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:10.272180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] MFA Rules not processed for user `801928d69b9a4c0688ed637ec6fa149c`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:10.273932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:10.313558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b91ef-2063-4602-b786-0f28e372bbbd service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:10.320201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:10.320851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:10.321132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:10.321406 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] PATH_INFO: `/v3/users/41ee5ebabbac4138bff8bb1728256e02` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:10.323873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ee5ebabbac4138bff8bb1728256e02)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:10.326468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:10.401953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1359/2712] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:10.408748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ee5ebabbac4138bff8bb1728256e02'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:10.418468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:10.422644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ee5ebabbac4138bff8bb1728256e02'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:10.425669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ee5ebabbac4138bff8bb1728256e02'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:10.426692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ee5ebabbac4138bff8bb1728256e02'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:10.434680 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef5a6d-d88c-441b-9eeb-7ef42197c8e3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:10.438733 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:10.439556 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:10.439832 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:10.439962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-2974b97c-1c78-4cd1-abe5-7faee27738c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ee5ebabbac4138bff8bb1728256e02'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:10.440124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:10.442111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:10.473473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1354/2713] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:10 2022] DELETE /identity/v3/users/41ee5ebabbac4138bff8bb1728256e02 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:10.478599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:10.478902 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:10.479166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:10.505514 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:10.507017 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:10.542275 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:10.543134 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:10.545845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:10.562927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1360/2714] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:10.645721 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1355/2715] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:10.651993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:10.654163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:10.726395 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:10.799793 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89761c-4355-4a8a-b51b-cfa7960e2690 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:10.938980 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:10.939933 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:10.943582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:10.943982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:11.009095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:11.021642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:11.105410 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1361/2716] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:11.719523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:11.721386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:11.740341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c1502-e6d9-42c4-8e5a-52ae3582ffc2 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:11.758152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:11.759075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:11.762741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:11.764762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] PATH_INFO: `/v3/projects/285c5c85192747cdb72dd884d474c320` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:11.769034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=285c5c85192747cdb72dd884d474c320)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:11.779782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:11.907871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285c5c85192747cdb72dd884d474c320', 'request_id': 'req-ed8f23a0-b1bb-487d-ab19-1192750c6b91'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:11.957631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285c5c85192747cdb72dd884d474c320', 'request_id': 'req-ed8f23a0-b1bb-487d-ab19-1192750c6b91'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:11.973926 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285c5c85192747cdb72dd884d474c320', 'request_id': 'req-ed8f23a0-b1bb-487d-ab19-1192750c6b91'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.016052 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] The token cache is being invalidate because project 285c5c85192747cdb72dd884d474c320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:12.016742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.020727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.027000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1356/2717] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:11 2022] DELETE /identity/v3/projects/285c5c85192747cdb72dd884d474c320 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:12.027536 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:12.045911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e421-96db-4f82-a48f-c4097a09bafe service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:12.204455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:12.207922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:12.211937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:12.212379 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] PATH_INFO: `/v3/users/cef6f885f5b34a2898746e4243e78b0b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:12.214421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef6f885f5b34a2898746e4243e78b0b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:12.219713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:12.411352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef6f885f5b34a2898746e4243e78b0b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.438293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef6f885f5b34a2898746e4243e78b0b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.471962 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef6f885f5b34a2898746e4243e78b0b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.473369 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef6f885f5b34a2898746e4243e78b0b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.489140 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c851c2f-5a16-466f-96fe-f912a99e34fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef6f885f5b34a2898746e4243e78b0b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:12.500968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1362/2718] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:12 2022] DELETE /identity/v3/users/cef6f885f5b34a2898746e4243e78b0b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:13.178870 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:13.180953 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:13.213398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f23a0-b1bb-487d-ab19-1192750c6b91 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:13.333444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:13.334522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:13.335019 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:13.335511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] PATH_INFO: `/v3/projects/c6110c04a67b4c26835160b622aede96` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:13.337614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6110c04a67b4c26835160b622aede96)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:13.340110 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:13.427962 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6110c04a67b4c26835160b622aede96', 'request_id': 'req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:13.436398 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:13.444301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6110c04a67b4c26835160b622aede96', 'request_id': 'req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:13.453207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6110c04a67b4c26835160b622aede96', 'request_id': 'req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:13.472563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] The token cache is being invalidate because project c6110c04a67b4c26835160b622aede96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:13.472792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:13.473469 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-7ec1aac1-e89d-4ace-8c0a-6ccf9061ac8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:13.493442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1357/2719] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:13 2022] DELETE /identity/v3/projects/c6110c04a67b4c26835160b622aede96 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:13.504031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9877-e514-4b0b-958b-af85718870f2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:13.504031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9877-e514-4b0b-958b-af85718870f2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:13.504031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9877-e514-4b0b-958b-af85718870f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:13.513618 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:13.546642 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a8dc9877-e514-4b0b-958b-af85718870f2 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:13.547667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a8dc9877-e514-4b0b-958b-af85718870f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:13.700372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1363/2720] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:13.709016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:13.709360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:13.709599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:13.740068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:13.740429 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:13.740769 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:13.761056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:13.762691 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:13.776793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1358/2721] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:13.780327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] MFA Rules not processed for user `b1e8669f366f47e0938ed3a4a1024081`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:13.780867 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:13.782104 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:13.792036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6a0d5-d76e-4815-91cf-7be118ef7206 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:13.798686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:13.799789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:13.799789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:13.800170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:13.801085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:13.801955 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:13.803204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:13.820522 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1359/2722] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:53:13.821690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:13.833825 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5eb3-e52d-423b-981d-cea269d2de31 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:13.842853 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:13.844733 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:13.845229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:13.846254 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:13.847515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:13.850700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:13.889502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1364/2723] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:13.903862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1360/2724] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:13 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:13.908854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:13.909881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:13.913986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:14.013243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90333b2-9df9-4168-89bd-44360d25f543 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.023164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.023164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.023164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.023164 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.025083 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.028151 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.072556 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1361/2725] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:13 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:14.080449 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.094952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ab580-f904-46fc-84e1-3837b615b197 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.102821 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.103497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.103778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.104186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.104857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.106612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.132392 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1362/2726] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:14.133970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.154144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ada7f-f78f-411d-b1d4-c27075dcf855 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.159510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.160529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.160529 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.160803 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.161438 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.163054 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.168696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:14.179448 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201f4e-a1cb-48f3-a8ab-9e6e75f63a91 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.184156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1363/2727] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:14.188374 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.203480 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850336c5-207f-4843-91d4-4a16e085814a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.218135 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a77c38-a486-4ec9-8729-9fc0c4a16410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.218920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a77c38-a486-4ec9-8729-9fc0c4a16410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.219215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a77c38-a486-4ec9-8729-9fc0c4a16410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.219551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a77c38-a486-4ec9-8729-9fc0c4a16410 admin admin] PATH_INFO: `/v3/projects/9f4a61ef70834ced9bc4c0d0b4984fb0/users/ee81b832859547aa82c7e552547cc3c1/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.242304 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a77c38-a486-4ec9-8729-9fc0c4a16410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4a61ef70834ced9bc4c0d0b4984fb0, user_id=ee81b832859547aa82c7e552547cc3c1, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.246421 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a77c38-a486-4ec9-8729-9fc0c4a16410 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.273021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1364/2728] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:14 2022] PUT /identity/v3/projects/9f4a61ef70834ced9bc4c0d0b4984fb0/users/ee81b832859547aa82c7e552547cc3c1/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:14.275061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.275791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.276666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.277001 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.279115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:14.289356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.290380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.292409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.348576 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] MFA Rules not processed for user `ee81b832859547aa82c7e552547cc3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:14.350593 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:14.419028 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.422553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:14.427133 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.453689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1365/2729] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:14.477633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1365/2730] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:14.482293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.494279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf6bee-27be-46c4-b5ac-75eb10c280d5 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.568701 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.569388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.569565 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.569992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.571561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.574919 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.616557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1366/2731] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:14 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:14.618094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.633020 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125f48a-8719-4d1b-aaa2-4c3f4f009329 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.708654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.709291 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.709589 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.709862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.710508 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.712533 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.736673 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1366/2732] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:14 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:14.740917 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.752568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbf91d-6de7-46a7-8c45-bd17014a4125 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.757446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.758071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.758361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.758635 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.759330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.761656 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.773825 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1367/2733] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:14.778970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.790082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6d51e-fabf-4d7b-8c5e-592ce8a4c2dc admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.795314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.796161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.796509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.796840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] PATH_INFO: `/v3/projects/35ccf7b881df4f5ba0fb49d56214120e/users/a1de5dcce7b249a19ba4eb0309eb1057/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.815932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ccf7b881df4f5ba0fb49d56214120e, user_id=a1de5dcce7b249a19ba4eb0309eb1057, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.818665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.837444 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1367/2734] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:14 2022] PUT /identity/v3/projects/35ccf7b881df4f5ba0fb49d56214120e/users/a1de5dcce7b249a19ba4eb0309eb1057/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:14.842071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.857735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aedfea-18bc-4ff9-aad3-f9fad28571c6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:14.869211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:14.953455 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562576ea-b173-40f9-bb54-cc55a3d9c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:14.954267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562576ea-b173-40f9-bb54-cc55a3d9c2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:14.954645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562576ea-b173-40f9-bb54-cc55a3d9c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:14.955016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562576ea-b173-40f9-bb54-cc55a3d9c2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:14.955842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562576ea-b173-40f9-bb54-cc55a3d9c2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:14.957586 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562576ea-b173-40f9-bb54-cc55a3d9c2fa admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:14.973615 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1368/2735] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:14.976665 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:14.999073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106341-1327-4ff0-a497-cb2031d80029 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:15.022036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:15.023571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:15.023909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:15.024238 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] PATH_INFO: `/v3/projects/35ccf7b881df4f5ba0fb49d56214120e/users/a1de5dcce7b249a19ba4eb0309eb1057/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:15.026887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ccf7b881df4f5ba0fb49d56214120e, user_id=a1de5dcce7b249a19ba4eb0309eb1057, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:15.029668 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:15.048920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1368/2736] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:14 2022] PUT /identity/v3/projects/35ccf7b881df4f5ba0fb49d56214120e/users/a1de5dcce7b249a19ba4eb0309eb1057/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:15.060643 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:15.060861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:15.061029 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:15.099236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] MFA Rules not processed for user `a1de5dcce7b249a19ba4eb0309eb1057`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:15.100584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:15.197086 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1369/2737] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:15.211748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:15.214222 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:15.325324 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a5b05-5aea-42a5-aafa-0787c51702f7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:15.331229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:15.331386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:15.331682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:15.331966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:15.367973 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:15.454544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:15.458991 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:15.485024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1369/2738] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:15.520473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:15.525695 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:15.578953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fee22c-9cd3-4798-8b22-77ba4c7df169 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:15.599264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:15.600018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:15.600326 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:15.600598 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:15.605522 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:15.796802 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:15.799387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749c8de-5518-4915-a751-7280b951d3c1 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:15.821887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1370/2739] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:18.409483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:18.491118 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d8819-216c-4460-b07d-8c60ae8d65e8 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:18.495968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:18.496599 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:18.496876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:18.497145 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] PATH_INFO: `/v3/users/ee81b832859547aa82c7e552547cc3c1` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:18.498295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee81b832859547aa82c7e552547cc3c1)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:18.500337 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:18.551988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee81b832859547aa82c7e552547cc3c1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:18.560395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee81b832859547aa82c7e552547cc3c1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:18.561267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee81b832859547aa82c7e552547cc3c1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:18.561716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee81b832859547aa82c7e552547cc3c1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:18.569850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee81b832859547aa82c7e552547cc3c1'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:18.588401 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1370/2740] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:18 2022] DELETE /identity/v3/users/ee81b832859547aa82c7e552547cc3c1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:18.593674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:18.594191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:18.594191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:18.614529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:18.615851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:18.642348 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:18.680399 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1371/2741] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:18.690223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:18.691978 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:18.738966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:18.755836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c8096-a5ba-48b6-bf53-963c99da498a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:18.759578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:18.760689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:18.761004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:18.761305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:18.771466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:18.773049 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:18.784132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1371/2742] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:19.156370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:19.158064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:19.186085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c37201-57e9-4bbb-8468-922f9fa2a0b6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:19.192301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:19.193058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:19.193383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:19.193588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] PATH_INFO: `/v3/projects/9f4a61ef70834ced9bc4c0d0b4984fb0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:19.194765 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4a61ef70834ced9bc4c0d0b4984fb0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:19.197074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:19.250848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4a61ef70834ced9bc4c0d0b4984fb0', 'request_id': 'req-4c237412-d5bf-47cc-847f-82a5d4bc4366'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.264449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4a61ef70834ced9bc4c0d0b4984fb0', 'request_id': 'req-4c237412-d5bf-47cc-847f-82a5d4bc4366'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.273212 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4a61ef70834ced9bc4c0d0b4984fb0', 'request_id': 'req-4c237412-d5bf-47cc-847f-82a5d4bc4366'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.296816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] The token cache is being invalidate because project 9f4a61ef70834ced9bc4c0d0b4984fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:19.297266 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.298057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.300016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1372/2743] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:19 2022] DELETE /identity/v3/projects/9f4a61ef70834ced9bc4c0d0b4984fb0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:19.304607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:19.314832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d735db-dd20-4d55-88eb-ae9681cccd8d service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:19.372189 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:19.372732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:19.373005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:19.373270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] PATH_INFO: `/v3/users/a1de5dcce7b249a19ba4eb0309eb1057` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:19.374501 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1de5dcce7b249a19ba4eb0309eb1057)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:19.376573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:19.426539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1de5dcce7b249a19ba4eb0309eb1057'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.434097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1de5dcce7b249a19ba4eb0309eb1057'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.434759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1de5dcce7b249a19ba4eb0309eb1057'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.435491 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1de5dcce7b249a19ba4eb0309eb1057'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.442530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1de5dcce7b249a19ba4eb0309eb1057'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.460607 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1372/2744] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:19 2022] DELETE /identity/v3/users/a1de5dcce7b249a19ba4eb0309eb1057 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:19.751758 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:19.753444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:19.775003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c237412-d5bf-47cc-847f-82a5d4bc4366 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:19.836042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:19.836683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:19.836965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:19.837237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] PATH_INFO: `/v3/projects/35ccf7b881df4f5ba0fb49d56214120e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:19.838657 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ccf7b881df4f5ba0fb49d56214120e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:19.840900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:19.890685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ccf7b881df4f5ba0fb49d56214120e', 'request_id': 'req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.893115 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:19.899607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ccf7b881df4f5ba0fb49d56214120e', 'request_id': 'req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.906238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ccf7b881df4f5ba0fb49d56214120e', 'request_id': 'req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.918380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] The token cache is being invalidate because project 35ccf7b881df4f5ba0fb49d56214120e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:19.918654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.919408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cce5290-e5f3-48a1-ba72-a5e0c8b607be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:19.921345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1373/2745] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:19 2022] DELETE /identity/v3/projects/35ccf7b881df4f5ba0fb49d56214120e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:29.456921 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:29.914594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=3,keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:30.303369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:30.304798 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:30.318141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dab4f-1c86-4cfc-93f1-2919867ac675 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:30.381266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:30.381899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:30.382179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:30.382442 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] PATH_INFO: `/v3/users/9afa99712d3346c2855bc79cb72ee4b3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:30.383594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afa99712d3346c2855bc79cb72ee4b3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:30.386087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:30.440014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afa99712d3346c2855bc79cb72ee4b3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:30.452486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afa99712d3346c2855bc79cb72ee4b3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:30.453354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afa99712d3346c2855bc79cb72ee4b3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:30.453965 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afa99712d3346c2855bc79cb72ee4b3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:30.461244 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afa99712d3346c2855bc79cb72ee4b3'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:30.479742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1373/2746] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:30 2022] DELETE /identity/v3/users/9afa99712d3346c2855bc79cb72ee4b3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:30.484355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:30.484667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:30.484941 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:30.504235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:30.505446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:30.566061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1374/2747] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:30.577007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:30.578430 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:30.638686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c66a-06f4-4364-af7d-f678941cdb89 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:30.642409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:30.643016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:30.643293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:30.643597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:30.653701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:30.655139 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:30.664767 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1374/2748] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:30.936621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:30.938105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:30.952097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79354c64-1eb0-4f20-8bcc-21077b446074 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:30.968549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:30.969577 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:30.969776 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:30.970044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] PATH_INFO: `/v3/projects/3c4eb8c79e1b451292edf5c59af5feca` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:30.971306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4eb8c79e1b451292edf5c59af5feca)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:30.973180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:31.030338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4eb8c79e1b451292edf5c59af5feca', 'request_id': 'req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.038236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4eb8c79e1b451292edf5c59af5feca', 'request_id': 'req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.044774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4eb8c79e1b451292edf5c59af5feca', 'request_id': 'req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.057916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] The token cache is being invalidate because project 3c4eb8c79e1b451292edf5c59af5feca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:31.058218 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.058877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.060539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1375/2749] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:30 2022] DELETE /identity/v3/projects/3c4eb8c79e1b451292edf5c59af5feca => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:31.064545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:31.079338 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9571ae8-4089-41a0-a66d-bf776172237c service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:31.102223 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:31.148994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:31.149936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:31.150204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:31.150483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] PATH_INFO: `/v3/users/6a2d990404214dd0840f3c98443babf9` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:31.151748 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2d990404214dd0840f3c98443babf9)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:31.153713 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:31.205497 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2d990404214dd0840f3c98443babf9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.213063 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2d990404214dd0840f3c98443babf9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.213778 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2d990404214dd0840f3c98443babf9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.214591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2d990404214dd0840f3c98443babf9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.224760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-c34ec669-7036-4631-a762-660a1a00e622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2d990404214dd0840f3c98443babf9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.245741 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1375/2750] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:31 2022] DELETE /identity/v3/users/6a2d990404214dd0840f3c98443babf9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:31.559327 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:31.560535 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:31.576655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b04c-0335-4cb6-a7f3-8c3145b82198 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:31.664073 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:31.664736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:31.665012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:31.665282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] PATH_INFO: `/v3/projects/91cf630022d0481fa4a358195dcc6995` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:31.666467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cf630022d0481fa4a358195dcc6995)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:31.668452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:31.708111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:31.725779 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cf630022d0481fa4a358195dcc6995', 'request_id': 'req-2d3aac62-7096-4dd2-8cd9-b405a503e786'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.732705 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cf630022d0481fa4a358195dcc6995', 'request_id': 'req-2d3aac62-7096-4dd2-8cd9-b405a503e786'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.738492 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cf630022d0481fa4a358195dcc6995', 'request_id': 'req-2d3aac62-7096-4dd2-8cd9-b405a503e786'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.750843 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] The token cache is being invalidate because project 91cf630022d0481fa4a358195dcc6995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:31.751179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.751814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2d3aac62-7096-4dd2-8cd9-b405a503e786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:31.753338 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1376/2751] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:31 2022] DELETE /identity/v3/projects/91cf630022d0481fa4a358195dcc6995 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:31.758178 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:31.758656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:31.758770 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:31.778537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:31.779763 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:31.837995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1376/2752] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:31.845865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:31.846146 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:31.846413 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:31.866644 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:31.868090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:31.877312 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1377/2753] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:31.881771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:31.892808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8375-0654-4552-aa64-fadde3015c7f None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:31.897692 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:31.898303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:31.898582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:31.898854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:31.899534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:31.899844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:31.901754 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:31.914654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1377/2754] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:53:31.919906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:31.929367 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86037-a499-424f-87bf-97c42a338282 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:31.935161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:31.935798 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:31.936075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:31.936341 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:31.936995 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:31.938927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:31.965458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1378/2755] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:31 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:31.969622 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:31.980948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e07d7-7fde-4ec0-bd3a-17142434304a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.012961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:32.056160 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.057686 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.057840 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.058073 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.058744 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.060775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.083048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1378/2756] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:31 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:32.087216 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.098182 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dbbd3-07f4-4226-9f19-48b4bf1d76f5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.109467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.110243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.112538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.112538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.112538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.114046 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.126698 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1379/2757] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:32.131451 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.143005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48c7d4-7121-4821-b3eb-a8bae431c2af admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.148537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac33f5-d84b-457f-87d7-7502f6e162ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.148907 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac33f5-d84b-457f-87d7-7502f6e162ed admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.149185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac33f5-d84b-457f-87d7-7502f6e162ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.149472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac33f5-d84b-457f-87d7-7502f6e162ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.150134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac33f5-d84b-457f-87d7-7502f6e162ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.152934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac33f5-d84b-457f-87d7-7502f6e162ed admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.165371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1379/2758] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:32.170961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.182105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23b7a2-4b37-41bf-ab29-9de605ba8d1f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.187174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.188051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.188330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.188786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] PATH_INFO: `/v3/projects/bc42685baa534da0b2d1dd5515065e9a/users/74f6f44d145d472aa96f359638000f5b/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.207562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc42685baa534da0b2d1dd5515065e9a, user_id=74f6f44d145d472aa96f359638000f5b, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.210453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.228950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1380/2759] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:32 2022] PUT /identity/v3/projects/bc42685baa534da0b2d1dd5515065e9a/users/74f6f44d145d472aa96f359638000f5b/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:32.239570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.239899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.240172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.268739 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] MFA Rules not processed for user `74f6f44d145d472aa96f359638000f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:32.269972 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:32.336007 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1380/2760] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:32.340624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.403916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ca20e-d88c-4dba-b295-cc565b1f0103 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.408564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.409169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.409444 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.409712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.410336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.412405 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.440846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1381/2761] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:32 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:32.445539 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.457158 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f5d5f-cc97-4491-bcdc-8acb08e83043 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.513618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.514233 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.514515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.514789 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.515447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.517395 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.539817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1381/2762] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:32 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:32.543835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.554561 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207ee3-059e-49e0-958b-2b87945197f8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.559960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.560568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.560845 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.561114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.561753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.563363 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.575003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1382/2763] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:32.579992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.591473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722009ea-bb21-484f-88b2-b120dc340548 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.596224 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.596887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.597163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.597433 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] PATH_INFO: `/v3/projects/51205372da8747da929d6ef81d2ffefd/users/83f38087e09f4d8a962bc003ad1f9ccc/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.613028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51205372da8747da929d6ef81d2ffefd, user_id=83f38087e09f4d8a962bc003ad1f9ccc, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.615753 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.629940 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1382/2764] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:32 2022] PUT /identity/v3/projects/51205372da8747da929d6ef81d2ffefd/users/83f38087e09f4d8a962bc003ad1f9ccc/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:32.634090 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.646223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2802827-b2b5-4106-af95-42bcc508f1e7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.668686 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:32.717960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30462e74-e007-4bdc-85ca-6d22d5044caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.718970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30462e74-e007-4bdc-85ca-6d22d5044caa admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.719252 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30462e74-e007-4bdc-85ca-6d22d5044caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.719553 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30462e74-e007-4bdc-85ca-6d22d5044caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.720514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30462e74-e007-4bdc-85ca-6d22d5044caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.722277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30462e74-e007-4bdc-85ca-6d22d5044caa admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.733599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1383/2765] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:53:32.739641 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.747534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:32.754542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa3421-1c0f-467e-9c14-b36a7581a366 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:32.761408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:32.762082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.762541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.762941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] PATH_INFO: `/v3/projects/51205372da8747da929d6ef81d2ffefd/users/83f38087e09f4d8a962bc003ad1f9ccc/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.767771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51205372da8747da929d6ef81d2ffefd, user_id=83f38087e09f4d8a962bc003ad1f9ccc, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:32.767771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:32.789142 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1383/2766] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:32 2022] PUT /identity/v3/projects/51205372da8747da929d6ef81d2ffefd/users/83f38087e09f4d8a962bc003ad1f9ccc/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:32.792622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:32.793808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:32.794089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:32.831317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] MFA Rules not processed for user `83f38087e09f4d8a962bc003ad1f9ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:32.832179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:32.924985 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1384/2767] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:32.933643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:32.935521 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:32.998990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad06e-b5ea-4a72-b60b-a0a139c73d1d admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:33.006087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:33.006700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:33.006977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:33.007247 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:33.017453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:33.018895 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:33.029385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1384/2768] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:34.097382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:34.099203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:34.193008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4efa0-5cb2-4f05-aaec-f2e4f69e0f8b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:34.198365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:34.199205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:34.199509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:34.199781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] PATH_INFO: `/v3/users/4e7b92a78a0e4389a7fe6e54e1e33a25` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:34.200911 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7b92a78a0e4389a7fe6e54e1e33a25)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:34.202821 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:34.255211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7b92a78a0e4389a7fe6e54e1e33a25'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.256421 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:34.269992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7b92a78a0e4389a7fe6e54e1e33a25'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.278138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7b92a78a0e4389a7fe6e54e1e33a25'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.278922 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7b92a78a0e4389a7fe6e54e1e33a25'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.290177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7b92a78a0e4389a7fe6e54e1e33a25'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.298161 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1385/2769] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:34 2022] DELETE /identity/v3/users/4e7b92a78a0e4389a7fe6e54e1e33a25 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:34.520747 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:34.537653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad0359-80c2-4335-adc4-dfa843f0f3ae service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:34.634066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:34.634927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:34.635207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:34.635511 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] PATH_INFO: `/v3/projects/1257196b4115437cb5822efb45e12c59` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:34.636661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1257196b4115437cb5822efb45e12c59)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:34.638594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:34.647549 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:34.649647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:34.667537 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c007b-be72-4269-98e4-f2e62ecb03db admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:34.673044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:34.673833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:34.674336 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:34.674621 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] PATH_INFO: `/v3/users/74f6f44d145d472aa96f359638000f5b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:34.676102 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f6f44d145d472aa96f359638000f5b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:34.678045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:34.695648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1257196b4115437cb5822efb45e12c59', 'request_id': 'req-6c5528ed-ebe5-4277-8b44-579e053359fb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.705059 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1257196b4115437cb5822efb45e12c59', 'request_id': 'req-6c5528ed-ebe5-4277-8b44-579e053359fb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.712735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1257196b4115437cb5822efb45e12c59', 'request_id': 'req-6c5528ed-ebe5-4277-8b44-579e053359fb'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.725711 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] The token cache is being invalidate because project 1257196b4115437cb5822efb45e12c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:34.725996 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.726644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.728215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1385/2770] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:34 2022] DELETE /identity/v3/projects/1257196b4115437cb5822efb45e12c59 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:34.732039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f6f44d145d472aa96f359638000f5b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.733296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:34.734735 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:34.745873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f6f44d145d472aa96f359638000f5b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.753904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f6f44d145d472aa96f359638000f5b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.754692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f6f44d145d472aa96f359638000f5b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.763077 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:34.763789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c0bc49ef-929e-4f38-958b-61352daf5e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f6f44d145d472aa96f359638000f5b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.770827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1386/2771] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:34 2022] DELETE /identity/v3/users/74f6f44d145d472aa96f359638000f5b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:34.776454 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:34.776965 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:34.777572 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:34.798805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:34.800267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:34.810751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5528ed-ebe5-4277-8b44-579e053359fb admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:34.862983 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1387/2772] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:34.874221 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:34.875692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:34.875892 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:34.876547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:34.876791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:34.877056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] PATH_INFO: `/v3/users/801928d69b9a4c0688ed637ec6fa149c` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:34.878208 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=801928d69b9a4c0688ed637ec6fa149c)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:34.880193 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:34.922165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:34.936270 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801928d69b9a4c0688ed637ec6fa149c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.941183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45b8e5-a1c3-4abe-81ff-71e8099f5a4e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:34.944647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801928d69b9a4c0688ed637ec6fa149c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.945472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801928d69b9a4c0688ed637ec6fa149c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.945741 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:34.946317 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:34.946558 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801928d69b9a4c0688ed637ec6fa149c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.946696 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:34.946864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:34.952672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:34.955555 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801928d69b9a4c0688ed637ec6fa149c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:34.966472 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:34.968018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:34.974951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1386/2773] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:34 2022] DELETE /identity/v3/users/801928d69b9a4c0688ed637ec6fa149c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:34.979743 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1388/2774] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:35.202515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.206939 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:35.283475 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:35.305675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479ea8b-0ff8-4484-8d42-762ea2be66d5 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.312333 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.313156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.313658 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.314724 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] PATH_INFO: `/v3/projects/4eec172a045a4686b7d633cffca2965e` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.316271 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eec172a045a4686b7d633cffca2965e)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.318518 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.331202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.332905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:35.348623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b2fde-9caf-4348-b563-8d66f534309d service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.359931 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.360989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.361366 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.361781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] PATH_INFO: `/v3/projects/bc42685baa534da0b2d1dd5515065e9a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.363070 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc42685baa534da0b2d1dd5515065e9a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.365377 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.401954 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eec172a045a4686b7d633cffca2965e', 'request_id': 'req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.412171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eec172a045a4686b7d633cffca2965e', 'request_id': 'req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.420427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eec172a045a4686b7d633cffca2965e', 'request_id': 'req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.433781 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc42685baa534da0b2d1dd5515065e9a', 'request_id': 'req-3a2e1354-40be-49a9-8984-6d978bdc169b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.437144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] The token cache is being invalidate because project 4eec172a045a4686b7d633cffca2965e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:35.437509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.438305 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8b77d66d-e159-4e69-9212-0e6c1c5a0bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.440286 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1387/2775] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:35 2022] DELETE /identity/v3/projects/4eec172a045a4686b7d633cffca2965e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:35.444193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc42685baa534da0b2d1dd5515065e9a', 'request_id': 'req-3a2e1354-40be-49a9-8984-6d978bdc169b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.446445 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf0c83-a676-4d90-8ee6-6636cc4a92af None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.447701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf0c83-a676-4d90-8ee6-6636cc4a92af None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.448182 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf0c83-a676-4d90-8ee6-6636cc4a92af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.451356 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc42685baa534da0b2d1dd5515065e9a', 'request_id': 'req-3a2e1354-40be-49a9-8984-6d978bdc169b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.468339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] The token cache is being invalidate because project bc42685baa534da0b2d1dd5515065e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:35.468655 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.469357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.470914 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1389/2776] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:35 2022] DELETE /identity/v3/projects/bc42685baa534da0b2d1dd5515065e9a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:35.475129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.475931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-95bf0c83-a676-4d90-8ee6-6636cc4a92af None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:35.477215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-95bf0c83-a676-4d90-8ee6-6636cc4a92af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:35.480541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:35.558353 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1388/2777] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:35.566331 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.566653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.566890 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.574928 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e1354-40be-49a9-8984-6d978bdc169b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.583629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.584515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.584791 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.585059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] PATH_INFO: `/v3/users/83f38087e09f4d8a962bc003ad1f9ccc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.586230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f38087e09f4d8a962bc003ad1f9ccc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.588901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.589032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:35.590283 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:35.598565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1389/2778] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:35.602967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.613172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f06f6-f49e-4096-9905-65e0eb6a0306 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.618880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.619575 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.620012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.620411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.621048 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:35.621578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.623542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.641950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1390/2779] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:53:35.646417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.654114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f38087e09f4d8a962bc003ad1f9ccc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.657499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6564b-3499-4cef-94ec-751d487f39e3 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.662538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.663166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.663472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.663982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.664386 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.666933 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.670912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f38087e09f4d8a962bc003ad1f9ccc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.673003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:35.683672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f38087e09f4d8a962bc003ad1f9ccc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.685445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f38087e09f4d8a962bc003ad1f9ccc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.697179 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f38087e09f4d8a962bc003ad1f9ccc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:35.703577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1391/2780] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:35 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:35.707264 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1390/2781] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:35 2022] DELETE /identity/v3/users/83f38087e09f4d8a962bc003ad1f9ccc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:35.707473 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.709132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:35.764560 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:35.801967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84647af8-b200-4dd8-8148-2161880625c7 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.886682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.886981 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.887580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.888236 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.888957 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.892185 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.925685 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1392/2782] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:35 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:35.926967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.942383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974a16a-d972-41d1-9a0a-291622f171c6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.947924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.948571 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.948924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.949085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.949721 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.951387 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:35.962582 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1391/2783] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:35.966911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:35.977417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625779-5b44-4e97-967c-1c9e8a08f139 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:35.988218 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:35.988862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:35.989138 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:35.989404 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:35.990061 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:35.992737 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.005217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1393/2784] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:36.011905 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.028093 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc9d5-853a-4d79-9f32-2cba2e1df136 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.045501 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfe167-8f58-44d1-bbb8-230a18cbf905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.046177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfe167-8f58-44d1-bbb8-230a18cbf905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.046456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfe167-8f58-44d1-bbb8-230a18cbf905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.046729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfe167-8f58-44d1-bbb8-230a18cbf905 admin admin] PATH_INFO: `/v3/projects/7c74d72722cd4b78879974da262ba92f/users/5aefa07173364325b77b5bce2c6f9b58/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.069589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfe167-8f58-44d1-bbb8-230a18cbf905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c74d72722cd4b78879974da262ba92f, user_id=5aefa07173364325b77b5bce2c6f9b58, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.073858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfe167-8f58-44d1-bbb8-230a18cbf905 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.087327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.092009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1392/2785] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:36 2022] PUT /identity/v3/projects/7c74d72722cd4b78879974da262ba92f/users/5aefa07173364325b77b5bce2c6f9b58/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:36.103797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.122656 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3de77-d66d-429f-abcd-f5c1cfafe4ca admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.170569 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.171114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.171629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.209539 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] MFA Rules not processed for user `5aefa07173364325b77b5bce2c6f9b58`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:36.211155 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.238998 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.239781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.243051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.243483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] PATH_INFO: `/v3/projects/51205372da8747da929d6ef81d2ffefd` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.245643 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51205372da8747da929d6ef81d2ffefd)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.247822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.299810 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1393/2786] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:36.300853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.302652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.305465 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51205372da8747da929d6ef81d2ffefd', 'request_id': 'req-311e540e-9930-4013-a7a7-694a209cb649'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.317854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51205372da8747da929d6ef81d2ffefd', 'request_id': 'req-311e540e-9930-4013-a7a7-694a209cb649'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.327012 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51205372da8747da929d6ef81d2ffefd', 'request_id': 'req-311e540e-9930-4013-a7a7-694a209cb649'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.327506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59f394-af6c-4f8c-9563-2b591d148066 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.328516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:36.343837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] The token cache is being invalidate because project 51205372da8747da929d6ef81d2ffefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:36.343837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.344590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.347549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1394/2787] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:36 2022] DELETE /identity/v3/projects/51205372da8747da929d6ef81d2ffefd => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:36.352089 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.354634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.394538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.395170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.395458 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.395734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] PATH_INFO: `/v3/users/875e4edde2834c8d9fd40cb219c4d2fd` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.396885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=875e4edde2834c8d9fd40cb219c4d2fd)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.398829 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.422554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e540e-9930-4013-a7a7-694a209cb649 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.426454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.427195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.427563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.427836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.429857 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.448416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:INSERT=2,keystone:DELETE=6,keystone:SELECT=92 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:36.456052 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875e4edde2834c8d9fd40cb219c4d2fd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.470108 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875e4edde2834c8d9fd40cb219c4d2fd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.479974 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875e4edde2834c8d9fd40cb219c4d2fd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.480628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875e4edde2834c8d9fd40cb219c4d2fd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.489138 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ec70d12-967b-495b-bcd0-e8051f0977e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875e4edde2834c8d9fd40cb219c4d2fd'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.495456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1394/2788] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:36 2022] DELETE /identity/v3/users/875e4edde2834c8d9fd40cb219c4d2fd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:36.497653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.499910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.500198 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.500447 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67769dc-03af-4923-932c-446ce592d50d service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.500557 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.512881 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1395/2789] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:36.522345 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:36.526330 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.616156 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1395/2790] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:36.617385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.617852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.617952 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.641808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:36.643044 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.652390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1396/2791] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:36.658166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.669380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96922cbc-78fe-4545-ac38-b525f78b9403 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.675094 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.676014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.676388 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.676749 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.677863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:36.677863 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.679595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.693784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1396/2792] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:53:36.699349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.712391 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e98e5-f3ed-4811-ab26-d2accfb98f70 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.719913 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.721141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.721618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.722107 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] PATH_INFO: `/v3/projects` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.723051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.724807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.726329 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.727197 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:36.751190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc266c-3ed3-4d21-b059-8ec618c53a1a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.769531 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.770232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.770627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.770786 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] PATH_INFO: `/v3/projects/bd71c6be7e9d4741b129fbc0e6fafad4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.772406 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd71c6be7e9d4741b129fbc0e6fafad4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.774225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1397/2793] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:36 2022] POST /identity/v3/projects => generated 407 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:36.774552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.780720 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.852541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd71c6be7e9d4741b129fbc0e6fafad4', 'request_id': 'req-20d84381-67eb-4a6f-9d95-9f251053d625'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.863861 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd71c6be7e9d4741b129fbc0e6fafad4', 'request_id': 'req-20d84381-67eb-4a6f-9d95-9f251053d625'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.867249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888e11-3cda-4edf-a3bf-4bee2d982546 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:36.874302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd71c6be7e9d4741b129fbc0e6fafad4', 'request_id': 'req-20d84381-67eb-4a6f-9d95-9f251053d625'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.890550 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] The token cache is being invalidate because project bd71c6be7e9d4741b129fbc0e6fafad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:36.890851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.891692 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:36.902639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:36.903518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1397/2794] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:36 2022] DELETE /identity/v3/projects/bd71c6be7e9d4741b129fbc0e6fafad4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:36.960181 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:36.961029 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:36.961468 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:36.961749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:36.962758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:36.965428 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:36.990148 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1398/2795] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:36 2022] POST /identity/v3/users => generated 304 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:36.995111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:36.996713 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:37.010906 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d84381-67eb-4a6f-9d95-9f251053d625 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.016459 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.017089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.017355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.017627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.018284 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.019972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.031498 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1398/2796] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:37.035852 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.048170 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f184f-85a6-405a-83d5-82b61784412a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.053082 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df69d7-a37d-480d-a48c-1d18727eba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.053942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df69d7-a37d-480d-a48c-1d18727eba3b admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.054226 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df69d7-a37d-480d-a48c-1d18727eba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.054495 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df69d7-a37d-480d-a48c-1d18727eba3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.055168 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df69d7-a37d-480d-a48c-1d18727eba3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.057639 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df69d7-a37d-480d-a48c-1d18727eba3b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.071198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1399/2797] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:37.078514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.089352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf8959-4dc8-475e-81a2-208aa5584852 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.094885 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.095815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.096039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.096376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] PATH_INFO: `/v3/projects/acbefc5f2ede4512a6091666aac3f670/users/009fd4c9c65a4729af12102393052721/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.112694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:37.124262 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbefc5f2ede4512a6091666aac3f670, user_id=009fd4c9c65a4729af12102393052721, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.127248 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.144141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1399/2798] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:37 2022] PUT /identity/v3/projects/acbefc5f2ede4512a6091666aac3f670/users/009fd4c9c65a4729af12102393052721/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:37.156597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.156919 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.157191 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.186791 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] MFA Rules not processed for user `009fd4c9c65a4729af12102393052721`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:37.188032 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:37.255629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1400/2799] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:37 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:37.260124 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.325016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211707b-d6e3-4bcd-8ce5-b95d14274fc6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.330481 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.331145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.331446 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.331723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.332400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.334706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.365716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1400/2800] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:37 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:37.366974 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.380472 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad64cb-6ea6-451d-85a2-3aa742853768 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.446119 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.446728 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.447003 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.447923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.448566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.450486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.473400 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1401/2801] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:37 2022] POST /identity/v3/users => generated 311 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:37.477815 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.489095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a0715-7b78-4e74-a7b7-4c33ee53a40b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.493960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.494893 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.495400 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.495730 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.496368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.497972 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.509018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1401/2802] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:37.514112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.524423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53c33-0faa-4528-ac22-b6fff41609a9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.529635 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.530292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.530565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.530829 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] PATH_INFO: `/v3/projects/c419e497db6549a793225f3ddae62259/users/f889842db1ce445b84fe6b009c3deb0e/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.549596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c419e497db6549a793225f3ddae62259, user_id=f889842db1ce445b84fe6b009c3deb0e, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.552654 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.563039 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:37.569051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1402/2803] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:37 2022] PUT /identity/v3/projects/c419e497db6549a793225f3ddae62259/users/f889842db1ce445b84fe6b009c3deb0e/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:37.573684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.583568 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0476888-ba45-46e3-8e2b-b74f88831da6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.641544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf1382-ef17-4048-9f5d-50e3d7a89e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.642160 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf1382-ef17-4048-9f5d-50e3d7a89e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.642438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf1382-ef17-4048-9f5d-50e3d7a89e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.642716 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf1382-ef17-4048-9f5d-50e3d7a89e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.643361 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf1382-ef17-4048-9f5d-50e3d7a89e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.644990 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf1382-ef17-4048-9f5d-50e3d7a89e97 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.655970 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1402/2804] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:37.660312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.672278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bf75d-5a1c-4eda-a7f9-1c39c9e7d96a admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.676956 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.677618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.677901 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.678172 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] PATH_INFO: `/v3/projects/c419e497db6549a793225f3ddae62259/users/f889842db1ce445b84fe6b009c3deb0e/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.680114 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c419e497db6549a793225f3ddae62259, user_id=f889842db1ce445b84fe6b009c3deb0e, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.683746 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.694674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1403/2805] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:37 2022] PUT /identity/v3/projects/c419e497db6549a793225f3ddae62259/users/f889842db1ce445b84fe6b009c3deb0e/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:37.707032 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.707313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.707619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.736099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] MFA Rules not processed for user `f889842db1ce445b84fe6b009c3deb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:37.737382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:37.787992 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:37.810604 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1403/2806] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:37 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:37.821349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:37.823780 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:37.889766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce336ec-9d5e-4e37-9cad-e59e67add38b admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:37.893578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:37.894228 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:37.894499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:37.894762 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:37.904648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:37.906100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:37.917013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1404/2807] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:37 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:38.093019 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:38.104308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0c7da-bd97-45ab-9c03-572b4c89ecd9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:38.108836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209dc30-f274-41e3-b928-37e9ebc458d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:38.112105 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209dc30-f274-41e3-b928-37e9ebc458d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:38.112385 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209dc30-f274-41e3-b928-37e9ebc458d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:38.112708 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209dc30-f274-41e3-b928-37e9ebc458d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:38.183003 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209dc30-f274-41e3-b928-37e9ebc458d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:38.184493 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209dc30-f274-41e3-b928-37e9ebc458d1 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:38.193926 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1404/2808] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:38 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:47.911866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:48.189734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=34 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:48.269894 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:48.271295 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:48.340526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112bf3-6117-4073-8c6e-d34a7b278ef6 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:48.345310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:48.345976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:48.346273 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:48.346537 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] PATH_INFO: `/v3/users/5aefa07173364325b77b5bce2c6f9b58` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:48.347707 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aefa07173364325b77b5bce2c6f9b58)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:48.349631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:48.396856 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aefa07173364325b77b5bce2c6f9b58'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:48.404425 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aefa07173364325b77b5bce2c6f9b58'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:48.405081 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aefa07173364325b77b5bce2c6f9b58'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:48.405700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aefa07173364325b77b5bce2c6f9b58'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:48.413278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aefa07173364325b77b5bce2c6f9b58'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:48.431509 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1405/2809] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:48 2022] DELETE /identity/v3/users/5aefa07173364325b77b5bce2c6f9b58 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:48.436316 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:48.436599 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:48.436867 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:48.456447 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:48.457596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:48.517632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1405/2810] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:48.528098 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:48.529585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:48.594385 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1b85-2a8b-4b3b-b811-0c4469369675 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:48.598516 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63695-dc26-4089-8e24-f533777a1284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:48.599136 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63695-dc26-4089-8e24-f533777a1284 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:48.599409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63695-dc26-4089-8e24-f533777a1284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:48.599701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63695-dc26-4089-8e24-f533777a1284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:48.610166 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63695-dc26-4089-8e24-f533777a1284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:48.612112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63695-dc26-4089-8e24-f533777a1284 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:48.635727 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1406/2811] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:48.922529 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:48.924125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:48.938329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60241f-6f0c-44e3-b42f-2a5ed4791e6b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:48.943255 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:48.943935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:48.944217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:48.944486 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] PATH_INFO: `/v3/projects/7c74d72722cd4b78879974da262ba92f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:48.945647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c74d72722cd4b78879974da262ba92f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:48.948268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:48.999232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c74d72722cd4b78879974da262ba92f', 'request_id': 'req-b37512b4-d218-41c6-99d0-602f80a4e864'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:49.007268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c74d72722cd4b78879974da262ba92f', 'request_id': 'req-b37512b4-d218-41c6-99d0-602f80a4e864'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:49.014018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c74d72722cd4b78879974da262ba92f', 'request_id': 'req-b37512b4-d218-41c6-99d0-602f80a4e864'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:49.026088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] The token cache is being invalidate because project 7c74d72722cd4b78879974da262ba92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:49.026372 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:49.027107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b37512b4-d218-41c6-99d0-602f80a4e864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:49.028638 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1406/2812] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:48 2022] DELETE /identity/v3/projects/7c74d72722cd4b78879974da262ba92f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:49.033563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802d370-1abf-4401-8ed6-c7f72b7b8162 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.034085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802d370-1abf-4401-8ed6-c7f72b7b8162 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.034085 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802d370-1abf-4401-8ed6-c7f72b7b8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.060364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-a802d370-1abf-4401-8ed6-c7f72b7b8162 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:49.061559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-a802d370-1abf-4401-8ed6-c7f72b7b8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:49.081742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:49.127291 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.127596 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.127865 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.133828 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1407/2813] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:49.142580 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.142873 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.143152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.148500 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] MFA Rules not processed for user `94533495d9ea4e489e7283dbf307ae66`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:49.149712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:49.162785 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:49.164045 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:49.173163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1408/2814] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:49.177756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.187650 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f6e0-f707-4ae9-9842-3a02476b9623 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.192250 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.192837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.193111 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.193376 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.194013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:49.194313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.195945 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.207416 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1407/2815] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:49.208014 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1409/2816] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 19:53:49.212370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.223620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c729-5654-4b40-b0ec-3420e0241add None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.229175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.229916 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.230095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.230519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.231002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.232959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.266229 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1408/2817] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:49 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:49.267347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.278034 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa5aa3-c393-4ee5-a906-c460eed259a8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.337095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.337759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.338207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.338653 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.339312 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.341315 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.364947 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1410/2818] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:49 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:49.369307 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.380075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5892e-c2ab-4ed4-aa2d-3453ec55d69e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.385950 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.386726 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.387008 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.387282 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.387977 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.389645 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.391201 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:49.403862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1409/2819] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:49.407782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.419299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f9195-a38d-464f-9a76-9d668bddceae admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.424417 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac634ae4-b028-4b0e-b3ee-a8aaac9028d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.425042 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac634ae4-b028-4b0e-b3ee-a8aaac9028d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.425323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac634ae4-b028-4b0e-b3ee-a8aaac9028d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.425596 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac634ae4-b028-4b0e-b3ee-a8aaac9028d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.426253 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac634ae4-b028-4b0e-b3ee-a8aaac9028d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.427924 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac634ae4-b028-4b0e-b3ee-a8aaac9028d7 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.439773 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1411/2820] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:49.443999 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.454809 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ad0e-31b4-49d6-822b-93bdd2b5ece5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.459706 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.460371 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.460649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.460921 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] PATH_INFO: `/v3/projects/65a2563441394312917070be74d5d7f8/users/9474a9898db04fdba69880f2b8519b7c/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.478064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a2563441394312917070be74d5d7f8, user_id=9474a9898db04fdba69880f2b8519b7c, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.480886 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.494833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1410/2821] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:49 2022] PUT /identity/v3/projects/65a2563441394312917070be74d5d7f8/users/9474a9898db04fdba69880f2b8519b7c/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:49.506549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.506759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.507036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.537424 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] MFA Rules not processed for user `9474a9898db04fdba69880f2b8519b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:49.538647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:49.607069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1412/2822] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:49.617912 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.620352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:49.693239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c043-5f7d-453d-83bf-f7dc17207106 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.698881 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606bd42-8ca2-434c-b383-4fe11534c137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.699813 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606bd42-8ca2-434c-b383-4fe11534c137 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.700191 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606bd42-8ca2-434c-b383-4fe11534c137 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.701023 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606bd42-8ca2-434c-b383-4fe11534c137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.711058 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606bd42-8ca2-434c-b383-4fe11534c137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.712814 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606bd42-8ca2-434c-b383-4fe11534c137 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.723373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1411/2823] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:49.839302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:49.841582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:49.857243 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:49.926394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83e0d8-6328-4c0b-97c9-b879967e4398 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:49.931677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:49.932434 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:49.932861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:49.933173 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] PATH_INFO: `/v3/users/009fd4c9c65a4729af12102393052721` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:49.934544 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009fd4c9c65a4729af12102393052721)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:49.936614 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:49.991884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009fd4c9c65a4729af12102393052721'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.000734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009fd4c9c65a4729af12102393052721'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.001440 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009fd4c9c65a4729af12102393052721'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.002146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009fd4c9c65a4729af12102393052721'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.012188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009fd4c9c65a4729af12102393052721'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.033278 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1413/2824] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:49 2022] DELETE /identity/v3/users/009fd4c9c65a4729af12102393052721 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:50.038222 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:50.038732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:50.039223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:50.068301 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:50.069789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:50.175132 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1412/2825] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:53:50.184790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:50.187483 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:50.263257 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79c5ee-f264-4997-ad60-378da7162b24 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:50.267096 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:50.267774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:50.268152 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:50.268287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:50.279334 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:50.280816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:50.290934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1414/2826] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:50.576040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:50.577732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:50.591722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e21f5d-3cf1-4c29-aaa4-e26ed17a173d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:50.596718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:50.597373 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:50.597633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:50.598088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] PATH_INFO: `/v3/projects/acbefc5f2ede4512a6091666aac3f670` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:50.599607 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acbefc5f2ede4512a6091666aac3f670)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:50.608654 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:50.661424 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acbefc5f2ede4512a6091666aac3f670', 'request_id': 'req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.676118 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acbefc5f2ede4512a6091666aac3f670', 'request_id': 'req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.683096 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acbefc5f2ede4512a6091666aac3f670', 'request_id': 'req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.690335 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:50.695746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] The token cache is being invalidate because project acbefc5f2ede4512a6091666aac3f670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:50.696202 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.697463 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.699079 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1413/2827] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:50 2022] DELETE /identity/v3/projects/acbefc5f2ede4512a6091666aac3f670 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:50.703213 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:50.713500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a658-bf59-4f2c-8966-91d319230667 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:50.749524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:50.774288 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:50.774845 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:50.775258 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:50.775611 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] PATH_INFO: `/v3/users/f889842db1ce445b84fe6b009c3deb0e` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:50.776944 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f889842db1ce445b84fe6b009c3deb0e)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:50.778893 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:50.833349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f889842db1ce445b84fe6b009c3deb0e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.842249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f889842db1ce445b84fe6b009c3deb0e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.843535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f889842db1ce445b84fe6b009c3deb0e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.844146 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f889842db1ce445b84fe6b009c3deb0e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.851541 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-125695d9-60b2-41ce-b587-2511687c1d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f889842db1ce445b84fe6b009c3deb0e'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:50.870761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1415/2828] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:50 2022] DELETE /identity/v3/users/f889842db1ce445b84fe6b009c3deb0e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:51.158960 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.160610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:51.176820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44b70b-b5b6-46a9-b4c0-3f2382f782c3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.238021 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.238717 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.239005 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.239279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] PATH_INFO: `/v3/projects/c419e497db6549a793225f3ddae62259` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.241163 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c419e497db6549a793225f3ddae62259)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.242901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.314453 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c419e497db6549a793225f3ddae62259', 'request_id': 'req-cad0526f-989d-466d-af15-101577eeb996'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:51.332122 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c419e497db6549a793225f3ddae62259', 'request_id': 'req-cad0526f-989d-466d-af15-101577eeb996'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:51.345720 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c419e497db6549a793225f3ddae62259', 'request_id': 'req-cad0526f-989d-466d-af15-101577eeb996'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:51.358563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] The token cache is being invalidate because project c419e497db6549a793225f3ddae62259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:51.358679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:51.359340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cad0526f-989d-466d-af15-101577eeb996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:51.360907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1414/2829] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:51 2022] DELETE /identity/v3/projects/c419e497db6549a793225f3ddae62259 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:53:51.366980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.367510 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.367602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.404989 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:51.406303 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:51.491347 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1416/2830] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:51.499389 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.499723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.500011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.522015 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:51.523274 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:51.536011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1415/2831] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:51.540078 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.555515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d382a-7a59-436d-b281-2979a7f19084 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.561909 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.562573 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.562855 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.563129 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.563850 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:51.564284 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.565824 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.582372 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1417/2832] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:53:51.587002 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.597518 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec421df5-5d67-485a-9eb8-c7fd8e31e123 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.606907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.609237 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.611544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.612071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.613404 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.616442 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.648691 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1416/2833] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:51 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:51.653775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.667765 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988a4c7-02d6-49cc-843f-6fb97d714156 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.754738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:51.781992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.782669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.782963 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.783235 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.783922 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.785929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.810790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1418/2834] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:51 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:51.815000 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.826474 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700e8e6-af24-48c1-90da-7c4048155013 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.831376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.832026 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.832489 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.832759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.833411 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.835084 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.846333 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1417/2835] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:51.850484 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.861849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a6d89-098c-42c8-a6e6-a4c54f2608f0 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.866664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18acc97-9de2-4498-98df-415a16c4a16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.867512 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18acc97-9de2-4498-98df-415a16c4a16e admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.867905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18acc97-9de2-4498-98df-415a16c4a16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.868307 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18acc97-9de2-4498-98df-415a16c4a16e admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.869851 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18acc97-9de2-4498-98df-415a16c4a16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.871948 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18acc97-9de2-4498-98df-415a16c4a16e admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.885632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1419/2836] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:51.889864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:51.900826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f84d4-5709-48d3-b47c-202afce953f3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:51.905690 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:51.906357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.906633 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.906957 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] PATH_INFO: `/v3/projects/b29b9322c3634a419488aeeca0fcb461/users/dd8a8ed551f24cada539bb37f037eed6/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.923083 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29b9322c3634a419488aeeca0fcb461, user_id=dd8a8ed551f24cada539bb37f037eed6, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:51.925907 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:51.940211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1418/2837] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:51 2022] PUT /identity/v3/projects/b29b9322c3634a419488aeeca0fcb461/users/dd8a8ed551f24cada539bb37f037eed6/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:51.952026 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:51.952308 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:51.952577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:51.981889 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] MFA Rules not processed for user `dd8a8ed551f24cada539bb37f037eed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:51.983122 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:52.054776 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1420/2838] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:52.066700 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:52.068505 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:52.098560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:52.143062 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c496de4-b975-4871-b8d1-81de80ba0e84 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:52.146909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bbfbf-9d0b-4fae-b20c-389d0a76b89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:52.147623 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bbfbf-9d0b-4fae-b20c-389d0a76b89f service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:52.147908 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bbfbf-9d0b-4fae-b20c-389d0a76b89f service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:52.148211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bbfbf-9d0b-4fae-b20c-389d0a76b89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:52.157263 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bbfbf-9d0b-4fae-b20c-389d0a76b89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:52.158709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bbfbf-9d0b-4fae-b20c-389d0a76b89f service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:52.171313 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1419/2839] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:55.377534 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:55.454144 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c5de8-4a5d-42b5-b9ac-837f457c708d None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:55.466549 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:55.467252 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:55.467561 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:55.467831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] PATH_INFO: `/v3/users/dd8a8ed551f24cada539bb37f037eed6` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:55.468993 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8a8ed551f24cada539bb37f037eed6)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:55.470931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:55.523787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8a8ed551f24cada539bb37f037eed6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:55.527526 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:55.534627 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8a8ed551f24cada539bb37f037eed6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:55.536409 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8a8ed551f24cada539bb37f037eed6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:55.537823 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8a8ed551f24cada539bb37f037eed6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:55.547910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8a8ed551f24cada539bb37f037eed6'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:55.567905 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1421/2840] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:53:55 2022] DELETE /identity/v3/users/dd8a8ed551f24cada539bb37f037eed6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:55.576022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:55.576943 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:55.577276 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:55.597811 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:55.599111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:55.661250 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1420/2841] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:55.671323 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:55.672882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:55.738206 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906cb00-27ef-4f36-9ba3-4df90ff400ca admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:55.741908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229a14d-dce5-44d8-9fc4-914ef8862cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:55.742517 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229a14d-dce5-44d8-9fc4-914ef8862cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:55.742795 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229a14d-dce5-44d8-9fc4-914ef8862cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:55.743065 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229a14d-dce5-44d8-9fc4-914ef8862cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:55.753351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229a14d-dce5-44d8-9fc4-914ef8862cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:55.754966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229a14d-dce5-44d8-9fc4-914ef8862cc0 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:55.765535 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1422/2842] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:56.082597 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.084967 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:56.115042 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7159f90-b2a8-414b-bd76-b40b0a60fadc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.121178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.121875 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.122243 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.122578 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] PATH_INFO: `/v3/projects/b29b9322c3634a419488aeeca0fcb461` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.123849 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29b9322c3634a419488aeeca0fcb461)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.125842 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.183091 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29b9322c3634a419488aeeca0fcb461', 'request_id': 'req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:56.194174 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29b9322c3634a419488aeeca0fcb461', 'request_id': 'req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:56.202624 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29b9322c3634a419488aeeca0fcb461', 'request_id': 'req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:56.217233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] The token cache is being invalidate because project b29b9322c3634a419488aeeca0fcb461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:53:56.217746 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:56.218513 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5f443c5d-23d1-4cf7-a63c-35dc8e31b8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:53:56.220663 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1421/2843] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:53:56 2022] DELETE /identity/v3/projects/b29b9322c3634a419488aeeca0fcb461 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:56.225466 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.225752 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.226056 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.254679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:56.256976 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:56.348090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1423/2844] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:56.356937 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.357232 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.357509 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.379830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:56.381515 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:56.391971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1422/2845] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:56.398591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.410794 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686bd08-43cf-440c-ac81-2ec60f837697 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.416588 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.417217 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.417499 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.417771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.418423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:53:56.418792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.420808 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.433917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1424/2846] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:53:56.438904 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.450099 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fae99-c414-425e-841b-e9269eb2e92a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.456546 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.457297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.457580 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.458011 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.458666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.460718 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.494844 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1423/2847] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:56.499523 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.510606 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0d073-d8c1-452b-b624-ae5b46b491e6 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.570642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:56.579454 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.580287 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.582364 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.582787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.583570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.585613 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.608595 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1425/2848] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:56.612964 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.624422 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393aace7-544a-4938-b1a7-e645ad9d291e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.630381 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.630991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.631272 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.631575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.632211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.633788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.645838 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1424/2849] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:56.650365 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.660999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5095f92-32bd-4f7b-a114-06f8d2e25fa9 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.667729 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13120973-09be-4bab-83db-09503eaf873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.668345 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120973-09be-4bab-83db-09503eaf873a admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.668618 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120973-09be-4bab-83db-09503eaf873a admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.668891 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120973-09be-4bab-83db-09503eaf873a admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.669536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13120973-09be-4bab-83db-09503eaf873a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.671410 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13120973-09be-4bab-83db-09503eaf873a admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.682980 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1426/2850] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:56.687285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.698820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b29a08-eccb-4153-927b-2961f8380402 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.704795 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.705177 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.705449 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.705805 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] PATH_INFO: `/v3/projects/221bca091ffd4a83b5a9f8163d0cb06b/users/2e22dc5ffa2f40de8a6542e2f472ada9/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.715924 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:56.725434 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221bca091ffd4a83b5a9f8163d0cb06b, user_id=2e22dc5ffa2f40de8a6542e2f472ada9, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.725901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.745993 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1425/2851] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:56 2022] PUT /identity/v3/projects/221bca091ffd4a83b5a9f8163d0cb06b/users/2e22dc5ffa2f40de8a6542e2f472ada9/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:56.756453 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.756736 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.757004 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.789245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] MFA Rules not processed for user `2e22dc5ffa2f40de8a6542e2f472ada9`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:56.791366 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:56.864302 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1427/2852] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:56.869022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.934510 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d608e47-18f1-4592-89c0-5cb726de85d2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:56.939235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:56.939817 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:56.940097 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:56.940365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:56.940987 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:56.942920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:56.972887 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1426/2853] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:56.976967 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:56.987880 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79a350-671e-4d99-946a-febf532921b5 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.052013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.052634 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.052915 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.053186 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.053832 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.055866 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.107632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1428/2854] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:56 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:53:57.111971 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.125555 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b85822-99ca-44b3-b711-a96faa571c3a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.131175 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.131848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.132125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.132393 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.133075 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.134788 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.146958 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1427/2855] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:57.151751 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.163732 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379a2e6-c1cf-4e01-b6dc-2151ac47c142 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.169925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576f056-bd39-4799-9b98-4a79d5ecf5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.170644 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f056-bd39-4799-9b98-4a79d5ecf5ce admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.170927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f056-bd39-4799-9b98-4a79d5ecf5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.171200 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f056-bd39-4799-9b98-4a79d5ecf5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.172013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576f056-bd39-4799-9b98-4a79d5ecf5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.173816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576f056-bd39-4799-9b98-4a79d5ecf5ce admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.185597 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1429/2856] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:57.190438 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.200605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe043ec-d47e-408d-8da4-b705a168c292 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.205346 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.206064 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.206349 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.206632 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] PATH_INFO: `/v3/projects/a756b1bbd9c54a8b8d99c711548f278f/users/8075c8aebcac4c0a96d2736b877c4712/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.223147 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a756b1bbd9c54a8b8d99c711548f278f, user_id=8075c8aebcac4c0a96d2736b877c4712, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.225853 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.239796 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1428/2857] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:57 2022] PUT /identity/v3/projects/a756b1bbd9c54a8b8d99c711548f278f/users/8075c8aebcac4c0a96d2736b877c4712/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:57.312363 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.312625 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.312898 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.343112 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] MFA Rules not processed for user `8075c8aebcac4c0a96d2736b877c4712`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:57.344914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:57.381958 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:57.419239 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1430/2858] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:57.424154 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.489478 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3e79-ddd9-4d25-aede-01265bfaa947 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.494744 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.495541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.495819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.496085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.496724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.498627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.527145 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1429/2859] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:53:57 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:57.531582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.547920 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51356eff-5477-47da-9b33-5d8c50c4eb1a None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.617936 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.618623 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.618900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.619225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.620057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.621994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.645207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1431/2860] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:53:57 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 19:53:57.649289 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.660440 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223774-adb2-4f13-bff0-bd566a717739 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.665823 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.666775 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.667089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.667380 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.668071 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.669984 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.681048 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1430/2861] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:57.685568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.695882 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6aefc-bf77-4199-afbe-84897500fe52 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.701137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.701790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.702069 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.702341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] PATH_INFO: `/v3/projects/27a14ce4ebd648dbb384d95db1d6ef4f/users/a17723c3b72e48a385716f46bccc10af/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.718503 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a14ce4ebd648dbb384d95db1d6ef4f, user_id=a17723c3b72e48a385716f46bccc10af, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.721225 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.734917 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1432/2862] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:57 2022] PUT /identity/v3/projects/27a14ce4ebd648dbb384d95db1d6ef4f/users/a17723c3b72e48a385716f46bccc10af/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:57.738951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.749169 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc045359-af00-4fe5-962b-3f5fabda6c4c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.757141 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:57.809443 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5a665-e07b-47fe-a0b9-64c3a2f41101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.810051 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5a665-e07b-47fe-a0b9-64c3a2f41101 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.810331 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5a665-e07b-47fe-a0b9-64c3a2f41101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.810605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5a665-e07b-47fe-a0b9-64c3a2f41101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.811236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5a665-e07b-47fe-a0b9-64c3a2f41101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.812862 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5a665-e07b-47fe-a0b9-64c3a2f41101 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.831889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1431/2863] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:53:57.836242 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:57.847698 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77989a-8bf6-42be-bf41-300fb8c24624 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:57.852296 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:57.853929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] REQUEST_METHOD: `PUT` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.853929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.853929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] PATH_INFO: `/v3/projects/27a14ce4ebd648dbb384d95db1d6ef4f/users/a17723c3b72e48a385716f46bccc10af/roles/dff984ec428b4b178510d79cce0a8cb3` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.855849 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a14ce4ebd648dbb384d95db1d6ef4f, user_id=a17723c3b72e48a385716f46bccc10af, role_id=dff984ec428b4b178510d79cce0a8cb3)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:57.858507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:57.874632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1433/2864] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:53:57 2022] PUT /identity/v3/projects/27a14ce4ebd648dbb384d95db1d6ef4f/users/a17723c3b72e48a385716f46bccc10af/roles/dff984ec428b4b178510d79cce0a8cb3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:53:57.880836 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:57.881238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:57.881485 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:57.913258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] MFA Rules not processed for user `a17723c3b72e48a385716f46bccc10af`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:53:57.914475 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:57.994830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1432/2865] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:53:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:53:58.006796 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:58.008578 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:53:58.071458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1b54f-a47b-44d3-a2a7-6bed1ee41eef admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:58.076066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:58.076879 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:58.077134 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:58.077435 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:58.119356 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:53:58.138524 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:58.140021 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:58.148908 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1434/2866] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:53:59.728634 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:53:59.738390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724ac74-d9fe-4f6f-9ec4-919e40d6c8f2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:53:59.741959 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920062bb-3bfb-432f-907c-11c619b4226a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:53:59.742560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920062bb-3bfb-432f-907c-11c619b4226a service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:53:59.742835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920062bb-3bfb-432f-907c-11c619b4226a service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:53:59.743195 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920062bb-3bfb-432f-907c-11c619b4226a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:53:59.809180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920062bb-3bfb-432f-907c-11c619b4226a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:53:59.810722 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920062bb-3bfb-432f-907c-11c619b4226a service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:53:59.821113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1433/2867] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:53:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:00.156167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:00.157760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:00.246982 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80d2e1-3d36-4714-afef-577ae60bdec9 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:00.252631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:00.253375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:00.253718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:00.254053 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] PATH_INFO: `/v3/users/9474a9898db04fdba69880f2b8519b7c` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:00.255314 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9474a9898db04fdba69880f2b8519b7c)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:00.257349 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:00.310693 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9474a9898db04fdba69880f2b8519b7c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.318999 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9474a9898db04fdba69880f2b8519b7c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.319701 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9474a9898db04fdba69880f2b8519b7c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.320301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9474a9898db04fdba69880f2b8519b7c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.328738 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9474a9898db04fdba69880f2b8519b7c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.346538 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1435/2868] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:54:00 2022] DELETE /identity/v3/users/9474a9898db04fdba69880f2b8519b7c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:00.351205 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:00.351538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:00.351816 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:00.375285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:00.376605 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:00.397570 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:00.398929 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:00.405144 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:00.442188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1434/2869] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:00.452745 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:00.454423 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:00.463547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bad9d-47f6-478f-8e92-610761107281 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:00.467726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:00.467931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:00.468246 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:00.468545 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:00.470429 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:00.483819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:00.515238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72111762-78a2-40d7-aa80-4afef447bf73 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:00.519376 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:00.520009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:00.520290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:00.520567 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:00.531450 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:00.532876 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:00.538740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:00.540205 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38aa63-c1e4-4a1c-9ab1-6696afd9e414 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:00.543641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1435/2870] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:00.550397 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1436/2871] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:00.826712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:00.828230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:00.842287 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49727da-c359-49c0-bb79-a5579d80740e service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:00.847209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:00.847996 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:00.848279 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:00.848566 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] PATH_INFO: `/v3/projects/65a2563441394312917070be74d5d7f8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:00.849667 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a2563441394312917070be74d5d7f8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:00.852347 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:00.907242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a2563441394312917070be74d5d7f8', 'request_id': 'req-0a11f85a-720b-4d1f-b8c7-5613322652e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.915392 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a2563441394312917070be74d5d7f8', 'request_id': 'req-0a11f85a-720b-4d1f-b8c7-5613322652e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.922883 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a2563441394312917070be74d5d7f8', 'request_id': 'req-0a11f85a-720b-4d1f-b8c7-5613322652e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.939734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] The token cache is being invalidate because project 65a2563441394312917070be74d5d7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:54:00.940012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.940674 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a11f85a-720b-4d1f-b8c7-5613322652e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:00.942297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1436/2872] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:54:00 2022] DELETE /identity/v3/projects/65a2563441394312917070be74d5d7f8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:00.948911 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:00.949267 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:00.949553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:00.971671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:00.972887 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:01.037091 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1437/2873] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:54:01.045579 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.045889 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.046159 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.072078 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:01.073630 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:01.081951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1437/2874] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:01.086293 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.096740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a313-03ed-4351-8159-e75c2abef9ff None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.105195 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.105810 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.106743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.107025 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.107775 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:54:01.108088 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.109709 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.127820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1438/2875] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:54:01.132484 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.143043 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcec7d0-dec5-4380-930a-5d5aa28183f9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.148039 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.148750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.149139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.149457 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.150206 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.152223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.185126 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1438/2876] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:54:01 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:54:01.190506 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.201229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d5068-0881-4ee1-a305-33c1f58ab870 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.262758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.263369 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.263675 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.263942 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.264583 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.266515 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.288790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1439/2877] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:54:01 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:54:01.293879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.304729 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000ac65-a800-4036-b001-b7934cefda83 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.310456 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.311059 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.311383 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.311689 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.312329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.313932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.324428 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1439/2878] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:54:01.326638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.328172 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.328655 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:01.338736 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2598003-a17b-4edd-986c-6f41e9a74fcd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.343452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.344061 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.344370 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.344650 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.345285 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.346874 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.360631 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1440/2879] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:54:01.361649 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.364456 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:01.373685 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f508ce-0305-435e-ac30-477d07750995 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.379057 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f50aa8-52e7-4bd6-903b-0adff3e83591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.379835 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f50aa8-52e7-4bd6-903b-0adff3e83591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.380114 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f50aa8-52e7-4bd6-903b-0adff3e83591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.380384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f50aa8-52e7-4bd6-903b-0adff3e83591 admin admin] PATH_INFO: `/v3/projects/da4a689e57aa4fc8a6bc9adc4891188a/users/8394b3206e4846cca84e9ba8a1c5baa4/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.382711 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:01.396708 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ed265-780d-48a8-b8b6-1889128599f1 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.399401 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f50aa8-52e7-4bd6-903b-0adff3e83591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4a689e57aa4fc8a6bc9adc4891188a, user_id=8394b3206e4846cca84e9ba8a1c5baa4, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.400783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.401543 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.401822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.402087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.402190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f50aa8-52e7-4bd6-903b-0adff3e83591 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.403884 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:01.417170 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1441/2880] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:54:01 2022] PUT /identity/v3/projects/da4a689e57aa4fc8a6bc9adc4891188a/users/8394b3206e4846cca84e9ba8a1c5baa4/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:54:01.429352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e54e41-68f5-41eb-a10d-40006eed280b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.429636 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e54e41-68f5-41eb-a10d-40006eed280b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.429909 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e54e41-68f5-41eb-a10d-40006eed280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.458451 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d3e54e41-68f5-41eb-a10d-40006eed280b None None] MFA Rules not processed for user `8394b3206e4846cca84e9ba8a1c5baa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:01.459740 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d3e54e41-68f5-41eb-a10d-40006eed280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:01.473411 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.474862 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.484797 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1440/2881] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 19:54:01.530833 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1442/2882] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:54:01.542163 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:01.543628 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:01.604783 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3f2b6-0a47-46eb-b606-e4e87a35a0d9 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:01.656826 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:01.657439 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:01.657718 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:01.658015 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:01.666837 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:01.668327 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:01.678820 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1441/2883] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:03.126487 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c457-4e72-4d7a-92bf-eddfec1f0e2d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:03.127125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c457-4e72-4d7a-92bf-eddfec1f0e2d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:03.127125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c457-4e72-4d7a-92bf-eddfec1f0e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:03.149068 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ff17c457-4e72-4d7a-92bf-eddfec1f0e2d None None] MFA Rules not processed for user `8394b3206e4846cca84e9ba8a1c5baa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:03.151785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ff17c457-4e72-4d7a-92bf-eddfec1f0e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:03.160056 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1443/2884] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:54:04.730781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:04.732326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:04.775298 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=100 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:04.809970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e25ca-6e47-4125-b9e4-84f72030aa30 service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:04.814530 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:04.815159 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:04.815458 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:04.815759 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] PATH_INFO: `/v3/users/2e22dc5ffa2f40de8a6542e2f472ada9` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:04.816927 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e22dc5ffa2f40de8a6542e2f472ada9)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:04.818839 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:04.866214 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e22dc5ffa2f40de8a6542e2f472ada9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:04.876161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e22dc5ffa2f40de8a6542e2f472ada9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:04.876836 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e22dc5ffa2f40de8a6542e2f472ada9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:04.877642 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e22dc5ffa2f40de8a6542e2f472ada9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:04.885294 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e22dc5ffa2f40de8a6542e2f472ada9'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:04.902827 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1442/2885] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:54:04 2022] DELETE /identity/v3/users/2e22dc5ffa2f40de8a6542e2f472ada9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:04.907290 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:04.907800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:04.907935 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:04.927479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:04.928608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:04.993275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1444/2886] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:05.003413 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:05.004844 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:05.062766 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e553a-3064-46f8-85c0-af271f81e9c4 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:05.066602 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:05.067245 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:05.067557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:05.067830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:05.077585 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:05.079031 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:05.088801 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1443/2887] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:05.412991 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:05.414595 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:05.437223 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba82f2-4434-4f61-9e34-8bcd9692577c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:05.443200 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:05.443898 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:05.444323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:05.444590 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] PATH_INFO: `/v3/projects/221bca091ffd4a83b5a9f8163d0cb06b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:05.445789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221bca091ffd4a83b5a9f8163d0cb06b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:05.449190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:05.501188 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221bca091ffd4a83b5a9f8163d0cb06b', 'request_id': 'req-104251fc-3f5e-4430-962f-fcb8c4ebfff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.511927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221bca091ffd4a83b5a9f8163d0cb06b', 'request_id': 'req-104251fc-3f5e-4430-962f-fcb8c4ebfff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.517210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:05.520045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221bca091ffd4a83b5a9f8163d0cb06b', 'request_id': 'req-104251fc-3f5e-4430-962f-fcb8c4ebfff4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.535033 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] The token cache is being invalidate because project 221bca091ffd4a83b5a9f8163d0cb06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:54:05.535322 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.536128 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.537666 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1445/2888] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:54:05 2022] DELETE /identity/v3/projects/221bca091ffd4a83b5a9f8163d0cb06b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:05.541858 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:05.551819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b535e9-a4e1-41dd-9fb0-3bad966a9e19 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:05.606925 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:05.614268 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:05.614950 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:05.615594 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:05.615916 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] PATH_INFO: `/v3/users/8075c8aebcac4c0a96d2736b877c4712` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:05.617171 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8075c8aebcac4c0a96d2736b877c4712)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:05.620232 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:05.668201 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8075c8aebcac4c0a96d2736b877c4712'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.675743 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8075c8aebcac4c0a96d2736b877c4712'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.676370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8075c8aebcac4c0a96d2736b877c4712'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.676966 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8075c8aebcac4c0a96d2736b877c4712'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.684559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8075c8aebcac4c0a96d2736b877c4712'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:05.703830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1444/2889] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:54:05 2022] DELETE /identity/v3/users/8075c8aebcac4c0a96d2736b877c4712 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:06.151344 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:06.152901 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:06.166357 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104251fc-3f5e-4430-962f-fcb8c4ebfff4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:06.234129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:06.234953 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:06.235297 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:06.235677 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] PATH_INFO: `/v3/projects/a756b1bbd9c54a8b8d99c711548f278f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:06.236961 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a756b1bbd9c54a8b8d99c711548f278f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:06.238986 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:06.297584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a756b1bbd9c54a8b8d99c711548f278f', 'request_id': 'req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.311563 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a756b1bbd9c54a8b8d99c711548f278f', 'request_id': 'req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.329576 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a756b1bbd9c54a8b8d99c711548f278f', 'request_id': 'req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.346778 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] The token cache is being invalidate because project a756b1bbd9c54a8b8d99c711548f278f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:54:06.347081 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.349408 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.352750 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1446/2890] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:54:06 2022] DELETE /identity/v3/projects/a756b1bbd9c54a8b8d99c711548f278f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:06.357180 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:06.358674 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:06.374123 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9bced-297c-45ac-9828-32a200f16ea8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:06.458983 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:06.459816 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:06.460097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:06.460371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] PATH_INFO: `/v3/users/a17723c3b72e48a385716f46bccc10af` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:06.461564 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17723c3b72e48a385716f46bccc10af)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:06.464740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:06.530690 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17723c3b72e48a385716f46bccc10af'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.538282 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17723c3b72e48a385716f46bccc10af'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.539062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17723c3b72e48a385716f46bccc10af'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.539730 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17723c3b72e48a385716f46bccc10af'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.548000 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8cb7cdc1-3644-44a4-881a-bd0f1d9063f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17723c3b72e48a385716f46bccc10af'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:06.560749 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:06.569756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1445/2891] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:54:06 2022] DELETE /identity/v3/users/a17723c3b72e48a385716f46bccc10af => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:06.922207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:06.923968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:06.938894 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a43ec-6abf-4c51-a3a2-685b0d5a73e2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.002409 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.003047 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.003329 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.003629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] PATH_INFO: `/v3/projects/27a14ce4ebd648dbb384d95db1d6ef4f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.004759 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a14ce4ebd648dbb384d95db1d6ef4f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.006672 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.034981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:07.083469 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a14ce4ebd648dbb384d95db1d6ef4f', 'request_id': 'req-daf5c582-1f88-4f17-ac18-cff2ab9886c1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:07.094669 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a14ce4ebd648dbb384d95db1d6ef4f', 'request_id': 'req-daf5c582-1f88-4f17-ac18-cff2ab9886c1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:07.102857 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a14ce4ebd648dbb384d95db1d6ef4f', 'request_id': 'req-daf5c582-1f88-4f17-ac18-cff2ab9886c1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:07.123193 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] The token cache is being invalidate because project 27a14ce4ebd648dbb384d95db1d6ef4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:54:07.123517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:07.124178 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-daf5c582-1f88-4f17-ac18-cff2ab9886c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:07.125882 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1447/2892] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:54:06 2022] DELETE /identity/v3/projects/27a14ce4ebd648dbb384d95db1d6ef4f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:07.130500 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.130792 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.131072 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.153666 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:07.154934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:07.218470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1446/2893] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:07.227207 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.227800 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.228209 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.249734 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:07.251009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:07.260742 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1448/2894] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:07.264979 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.275951 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e2a7-13e4-44f4-a70f-8cf1409f1cfe None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.281094 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.281742 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.282079 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.282299 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.283005 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81801) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 19:54:07.283309 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.285090 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.298830 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1447/2895] 10.222.0.55 () {58 vars in 1153 bytes} [Sat Sep 24 19:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 19:54:07.303641 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.313353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559a2e9-5238-4461-8504-a16c0ef79b02 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.318302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.318988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.319271 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.319818 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.320230 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.323018 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.355165 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1449/2896] 10.222.0.55 () {60 vars in 1153 bytes} [Sat Sep 24 19:54:07 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:54:07.359387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.369565 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a7b1c-8e9d-4f39-b050-a04ed33118d8 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.432723 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.433387 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.433664 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.433934 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] PATH_INFO: `/v3/users` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.434568 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.436551 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.461771 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1448/2897] 10.222.0.55 () {60 vars in 1144 bytes} [Sat Sep 24 19:54:07 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 19:54:07.466808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.478968 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7485-d663-4b9d-a6df-ec3b3c453177 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.486610 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.487235 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.487562 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.487827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.488468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.490060 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.501807 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1450/2898] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 19:54:07.505700 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.517782 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d5ab5-ce82-4858-97e8-15c0d08633de admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.523591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c022f7-a6a9-48b0-825e-9bceddcded8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.524341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c022f7-a6a9-48b0-825e-9bceddcded8f admin admin] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.525266 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c022f7-a6a9-48b0-825e-9bceddcded8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.525553 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c022f7-a6a9-48b0-825e-9bceddcded8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.526204 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c022f7-a6a9-48b0-825e-9bceddcded8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.527875 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c022f7-a6a9-48b0-825e-9bceddcded8f admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.548914 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1449/2899] 10.222.0.55 () {58 vars in 1122 bytes} [Sat Sep 24 19:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 19:54:07.553041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.565608 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccf20f-27b1-4649-a98f-95beee73dfd1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.581517 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.582932 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.583661 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.584129 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] PATH_INFO: `/v3/projects/6efc25cd128640cc8eb550d563e9489e/users/26c842cc144e442698b16838c589ebd2/roles/1abc6940fcf2429aab6c6fc960488040` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.602981 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6efc25cd128640cc8eb550d563e9489e, user_id=26c842cc144e442698b16838c589ebd2, role_id=1abc6940fcf2429aab6c6fc960488040)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.605820 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.623521 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1451/2900] 10.222.0.55 () {60 vars in 1483 bytes} [Sat Sep 24 19:54:07 2022] PUT /identity/v3/projects/6efc25cd128640cc8eb550d563e9489e/users/26c842cc144e442698b16838c589ebd2/roles/1abc6940fcf2429aab6c6fc960488040 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:07.633652 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1ec43-8166-4826-a5b7-5c9c6155c6d8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.635669 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1ec43-8166-4826-a5b7-5c9c6155c6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.636016 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1ec43-8166-4826-a5b7-5c9c6155c6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.666647 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-c6c1ec43-8166-4826-a5b7-5c9c6155c6d8 None None] MFA Rules not processed for user `26c842cc144e442698b16838c589ebd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:07.668010 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c6c1ec43-8166-4826-a5b7-5c9c6155c6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:07.719390 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:07.737802 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1450/2901] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:07.749480 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:07.751189 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:07.810684 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de41128-c943-46a9-9157-043fc8e0c739 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:07.814203 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:07.814801 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:07.815074 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:07.815340 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:07.824617 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:07.826035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:07.837723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1452/2902] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:17.734784 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=4 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:17.830920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:19.870354 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a8d62-80e6-484f-bcf3-6bb23bf4f77c None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:19.870900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a8d62-80e6-484f-bcf3-6bb23bf4f77c None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:19.870900 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a8d62-80e6-484f-bcf3-6bb23bf4f77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:19.901264 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-bd3a8d62-80e6-484f-bcf3-6bb23bf4f77c None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:19.902470 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-bd3a8d62-80e6-484f-bcf3-6bb23bf4f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:19.957408 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1451/2903] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:54:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 19:54:20.003982 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:20.005461 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:20.078085 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d818926-edba-4dee-9eda-da7cc8be3669 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:20.083035 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:20.083652 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:20.083927 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:20.084192 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:20.093180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:20.094755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:20.103139 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1453/2904] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:54:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:54:20.621577 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3ffa5-8eaa-4d89-830c-61dd75faa0a6 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:20.621861 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3ffa5-8eaa-4d89-830c-61dd75faa0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:20.622132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3ffa5-8eaa-4d89-830c-61dd75faa0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:20.641467 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-13f3ffa5-8eaa-4d89-830c-61dd75faa0a6 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:20.642682 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-13f3ffa5-8eaa-4d89-830c-61dd75faa0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:20.650968 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1452/2905] 10.222.0.55 () {58 vars in 967 bytes} [Sat Sep 24 19:54:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:20.665185 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:20.676998 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069be263-a20d-44b1-8cd1-a7808d019d3e service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:20.682183 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:20.683041 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:20.683325 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:20.683628 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:20.692382 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:20.694012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:20.704541 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1454/2906] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:54:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:54:20.936259 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c088c4-7dae-4112-854b-6e4f2d5f5bdb None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:20.936781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c088c4-7dae-4112-854b-6e4f2d5f5bdb None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:20.936883 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c088c4-7dae-4112-854b-6e4f2d5f5bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:20.956985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-b8c088c4-7dae-4112-854b-6e4f2d5f5bdb None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:20.958167 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b8c088c4-7dae-4112-854b-6e4f2d5f5bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:20.966179 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1453/2907] 10.222.0.55 () {58 vars in 1009 bytes} [Sat Sep 24 19:54:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:20.978647 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:20.990846 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f05e5-76fe-4447-a1e7-9e17818055bc service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:20.996024 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:20.996629 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:20.996910 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:20.997180 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:21.006009 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:21.007688 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:21.016879 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1455/2908] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:54:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:54:21.346383 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:21.346689 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:21.346961 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:21.366706 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] MFA Rules not processed for user `81fd5bd3dbf743d8b6c6545fa33afdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:21.368002 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:21.375133 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1454/2909] 10.222.0.55 () {58 vars in 967 bytes} [Sat Sep 24 19:54:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:21.387445 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:21.398136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40523d2-9165-4ecb-8f22-631a1c05d6df service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:21.404194 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068664a8-4569-4b47-80b5-fa3ba6eddda5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c72b7d859e954054848d405afa0b466f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:21.404735 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068664a8-4569-4b47-80b5-fa3ba6eddda5 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:21.405014 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068664a8-4569-4b47-80b5-fa3ba6eddda5 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:21.405281 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068664a8-4569-4b47-80b5-fa3ba6eddda5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:21.413812 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068664a8-4569-4b47-80b5-fa3ba6eddda5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:21.416512 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068664a8-4569-4b47-80b5-fa3ba6eddda5 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:21.425502 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1456/2910] 10.222.0.55 () {60 vars in 1384 bytes} [Sat Sep 24 19:54:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 19:54:23.550559 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:23.552062 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:23.621249 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2c83-ebf5-4d86-abf9-592609232f45 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:23.625726 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:23.626351 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:23.626629 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:23.626899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] PATH_INFO: `/v3/users/8394b3206e4846cca84e9ba8a1c5baa4` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:23.628051 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8394b3206e4846cca84e9ba8a1c5baa4)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:23.630001 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:23.683141 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8394b3206e4846cca84e9ba8a1c5baa4'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:23.691761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8394b3206e4846cca84e9ba8a1c5baa4'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:23.692378 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8394b3206e4846cca84e9ba8a1c5baa4'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:23.692986 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8394b3206e4846cca84e9ba8a1c5baa4'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:23.701846 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8394b3206e4846cca84e9ba8a1c5baa4'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:23.715306 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:23.726207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1455/2911] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:54:23 2022] DELETE /identity/v3/users/8394b3206e4846cca84e9ba8a1c5baa4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:23.732022 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:23.732594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:23.732594 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:23.755012 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:23.756343 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:23.828506 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1457/2912] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:23.838375 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:23.840326 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:23.906187 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a88a-2a59-4e79-a7c7-10efdf3ab537 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:23.910057 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:23.910667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:23.910941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:23.911207 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:23.921676 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:23.923131 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:23.942156 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1456/2913] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:24.135265 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:24.136739 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:24.149777 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529b3a6-20e5-4d34-bf8f-4c2e0a52bbe0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:24.158514 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:24.159321 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:24.159826 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:24.160190 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] PATH_INFO: `/v3/projects/da4a689e57aa4fc8a6bc9adc4891188a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:24.161468 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da4a689e57aa4fc8a6bc9adc4891188a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:24.163589 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:24.217888 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:24.221236 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da4a689e57aa4fc8a6bc9adc4891188a', 'request_id': 'req-d581abf4-5761-42eb-bc7e-2fda075e8158'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:24.229808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da4a689e57aa4fc8a6bc9adc4891188a', 'request_id': 'req-d581abf4-5761-42eb-bc7e-2fda075e8158'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:24.237789 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da4a689e57aa4fc8a6bc9adc4891188a', 'request_id': 'req-d581abf4-5761-42eb-bc7e-2fda075e8158'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:24.249753 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] The token cache is being invalidate because project da4a689e57aa4fc8a6bc9adc4891188a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:54:24.250040 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:24.250709 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d581abf4-5761-42eb-bc7e-2fda075e8158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:24.252238 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1458/2914] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:54:24 2022] DELETE /identity/v3/projects/da4a689e57aa4fc8a6bc9adc4891188a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 19:54:33.928563 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=28 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:34.246622 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:43.118038 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:43.119486 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:43.132941 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabdd2b-2a0d-4009-b124-22c1226cdfd0 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:43.193348 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:43.193985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:43.194265 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:43.194536 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] PATH_INFO: `/v3/users/26c842cc144e442698b16838c589ebd2` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:43.195904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c842cc144e442698b16838c589ebd2)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:43.197994 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:43.250761 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c842cc144e442698b16838c589ebd2'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.260812 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c842cc144e442698b16838c589ebd2'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.261423 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c842cc144e442698b16838c589ebd2'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.262024 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c842cc144e442698b16838c589ebd2'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.269750 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c842cc144e442698b16838c589ebd2'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.287582 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1457/2915] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 19:54:43 2022] DELETE /identity/v3/users/26c842cc144e442698b16838c589ebd2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:43.293016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:43.293544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:43.293544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:43.314217 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] MFA Rules not processed for user `818c3ae70a684477af4280e7bbdcea2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:54:43.315339 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:43.373784 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1459/2916] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:54:43.383681 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:43.385087 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:43.443764 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560978b4-ddb4-4d2b-b581-6e6a307acf44 admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:43.447482 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:43.448093 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:43.448371 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:43.448638 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:43.457648 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:43.459086 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:43.468932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1458/2917] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:54:43.739600 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:54:43.741027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:54:43.754662 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2eaf-2a51-4a46-acf8-11c5f39fc2f0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:54:43.760210 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:54:43.760851 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:54:43.761125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:54:43.761390 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] PATH_INFO: `/v3/projects/6efc25cd128640cc8eb550d563e9489e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:54:43.762533 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6efc25cd128640cc8eb550d563e9489e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:54:43.766125 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:54:43.815544 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6efc25cd128640cc8eb550d563e9489e', 'request_id': 'req-06b27ef2-488d-4247-8380-1666dc58ca11'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.824228 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6efc25cd128640cc8eb550d563e9489e', 'request_id': 'req-06b27ef2-488d-4247-8380-1666dc58ca11'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.830095 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6efc25cd128640cc8eb550d563e9489e', 'request_id': 'req-06b27ef2-488d-4247-8380-1666dc58ca11'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.841837 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] The token cache is being invalidate because project 6efc25cd128640cc8eb550d563e9489e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 19:54:43.842121 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.842757 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 19:54:43.844467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1460/2918] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 19:54:43 2022] DELETE /identity/v3/projects/6efc25cd128640cc8eb550d563e9489e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 19:54:53.464734 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:54:53.838780 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:55:58.961822 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:55:58.970557 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9952-c7b4-4a4b-9815-37557ad47508 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:55:59.018985 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:55:59.019609 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:55:59.019886 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:55:59.020150 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:55:59.021871 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:55:59.031313 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:55:59.032774 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:55:59.041336 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1459/2919] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:02.301592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:56:02.303111 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:02.316267 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b27ef2-488d-4247-8380-1666dc58ca11 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:56:02.320552 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:56:02.321153 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:02.321430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:02.321694 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:02.323523 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:02.395374 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:56:02.396858 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:56:02.408275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1461/2920] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:03.589632 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:56:03.591716 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:03.605137 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634c975-e56c-4e1c-bdcb-b623061170f1 service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:56:03.608793 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:56:03.609380 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:03.609661 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:03.609932 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:03.611677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:03.625631 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:56:03.627060 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f915d-ce54-40ed-9e9d-4df3a56456c9 service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:56:03.638075 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1460/2921] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:05.337693 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:56:05.339306 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:05.357785 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d1bbd-7fc8-4a4f-a3c5-ade823115fdd service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:56:05.367407 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:56:05.368302 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:05.368592 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:05.368878 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:05.370803 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:05.386920 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:56:05.388819 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:56:05.400873 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1462/2922] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:56:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:07.055095 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] REQUEST_METHOD: `POST` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:07.055677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:07.055677 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:07.074831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.auth.core [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] MFA Rules not processed for user `94533495d9ea4e489e7283dbf307ae66`. Rule list: `[]` (Enabled: `True`). {{(pid=81801) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:56:07.076066 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:07.140554 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1461/2923] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:56:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:56:07.472989 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:56:07.483847 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388f984-b6c9-4e04-bf27-e8c48364d78b service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:56:07.487830 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:56:07.488430 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:07.488712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:07.488988 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:07.491314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:07.501173 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:56:07.502588 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:56:07.511239 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1463/2924] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:56:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:08.981598 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:56:08.982995 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:09.058591 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6a7c7-1764-4b46-a2c9-84727a941cd8 None None] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:56:09.061301 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:56:09.061923 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:09.062198 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:09.062463 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:09.064831 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:09.079310 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:56:09.080970 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:56:09.092241 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1462/2925] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:17.507044 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=46 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:56:19.088547 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=89 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:56:27.153601 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:56:27.169620 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11bb81-f7c4-4d1b-bbfb-602d69425f4d service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:56:27.175365 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40904a-33ea-4541-9559-61850431d96c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:56:27.175997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40904a-33ea-4541-9559-61850431d96c service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:56:27.176275 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40904a-33ea-4541-9559-61850431d96c service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:56:27.176538 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40904a-33ea-4541-9559-61850431d96c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:56:27.178352 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-de40904a-33ea-4541-9559-61850431d96c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:56:27.191547 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40904a-33ea-4541-9559-61850431d96c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:56:27.193166 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40904a-33ea-4541-9559-61850431d96c service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:56:27.207135 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1464/2926] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:56:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:56:37.201728 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:58:13.302036 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:58:13.303603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:13.368394 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed028b5a-cc94-4b2b-8950-84a6e6e45541 service placement] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:58:13.372740 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93ea821819d462097b1e76ce1ffae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:58:13.373571 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:58:13.373815 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:58:13.374103 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:58:13.377040 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:13.390215 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:58:13.392013 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:58:13.402904 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1463/2927] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:58:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:58:17.075723 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:58:17.076323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:58:17.076323 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:58:17.096551 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] MFA Rules not processed for user `94533495d9ea4e489e7283dbf307ae66`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 19:58:17.097864 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:17.105107 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1465/2928] 10.222.0.55 () {58 vars in 958 bytes} [Sat Sep 24 19:58:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 19:58:18.418817 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:58:18.420357 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:18.435854 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c775-b96f-47a9-a718-b1d0550e397f service cinder] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:58:18.441590 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:58:18.442199 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:58:18.442479 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:58:18.442745 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:58:18.444552 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:18.457325 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:58:18.458977 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:58:18.470023 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1464/2929] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:58:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:58:27.090712 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:58:28.464988 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=32 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 19:58:36.229890 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 19:58:36.231497 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:36.249464 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b2bd5-614f-4bc6-af4f-9b9a24a573ee None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 19:58:36.255167 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f68f991883493dbf78c0bcfd827c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 19:58:36.255808 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 19:58:36.256088 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 19:58:36.256360 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 19:58:36.258113 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 19:58:36.270940 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 19:58:36.272573 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 19:58:36.283679 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1466/2930] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 19:58:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 19:58:46.280211 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:00:19.856161 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:19.867176 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d01cf-36a4-4798-a57f-8f4cf96e8b5d service placement] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:19.871132 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8702de7dc5fd4fdaa05449938d461de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:19.871931 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:19.872300 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:19.872699 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:19.874760 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:00:19.958633 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:19.961393 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:19.980028 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1465/2931] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:00:23.951564 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:23.963619 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbbe05-4450-4e7e-8113-9a553daf7287 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:23.969402 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:23.970016 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:23.970293 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:23.970560 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:23.972627 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:00:23.986136 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:23.987774 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:24.000006 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1467/2932] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 20:00:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:00:27.525756 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:27.548807 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3187f3-b7cc-4b2d-ab49-dea7f762b39a service neutron] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:27.559281 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:27.568128 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] REQUEST_METHOD: `GET` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:27.568619 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:27.569370 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:27.572819 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:00:27.600584 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:27.603368 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:00:27.606360 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:27.609899 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:27.620277 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63a121-05b8-4fc9-8ffc-04a761ea60f2 service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:27.625966 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:27.626702 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:27.627045 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:27.627479 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] PATH_INFO: `/v3/users/94533495d9ea4e489e7283dbf307ae66` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:27.628848 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94533495d9ea4e489e7283dbf307ae66)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:27.630902 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:27.635229 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1466/2933] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 20:00:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:00:27.701378 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94533495d9ea4e489e7283dbf307ae66'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:27.716452 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94533495d9ea4e489e7283dbf307ae66'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:27.727683 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94533495d9ea4e489e7283dbf307ae66'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:27.728604 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94533495d9ea4e489e7283dbf307ae66'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:27.741049 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94533495d9ea4e489e7283dbf307ae66'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:27.752355 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1468/2934] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 20:00:27 2022] DELETE /identity/v3/users/94533495d9ea4e489e7283dbf307ae66 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:00:27.982292 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:27.994566 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604edb5c-5b96-48e7-96e9-311af8a634d0 service placement] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:28.071562 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:28.072341 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:28.072758 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:28.073127 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] PATH_INFO: `/v3/projects/d308ae5684724251aa1011f269cf466a` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:28.075603 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d308ae5684724251aa1011f269cf466a)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:28.078084 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:28.138910 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd308ae5684724251aa1011f269cf466a', 'request_id': 'req-368e4ac8-f057-4a90-a05b-c33ea127ad6c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.150667 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd308ae5684724251aa1011f269cf466a', 'request_id': 'req-368e4ac8-f057-4a90-a05b-c33ea127ad6c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.159787 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd308ae5684724251aa1011f269cf466a', 'request_id': 'req-368e4ac8-f057-4a90-a05b-c33ea127ad6c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.179612 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] The token cache is being invalidate because project d308ae5684724251aa1011f269cf466a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:00:28.180140 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.181263 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.184344 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1467/2935] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 20:00:27 2022] DELETE /identity/v3/projects/d308ae5684724251aa1011f269cf466a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:00:28.191334 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:28.193895 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:00:28.218732 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bac36-2e43-4014-9466-68fc3ce6bfe6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:28.315827 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:28.316473 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:28.316755 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:28.317027 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] PATH_INFO: `/v3/users/b1e8669f366f47e0938ed3a4a1024081` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:28.318242 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e8669f366f47e0938ed3a4a1024081)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:28.320233 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:28.379724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e8669f366f47e0938ed3a4a1024081'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.392431 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:00:28.394997 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e8669f366f47e0938ed3a4a1024081'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.404516 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e8669f366f47e0938ed3a4a1024081'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.405384 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e8669f366f47e0938ed3a4a1024081'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.414467 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e8669f366f47e0938ed3a4a1024081'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.421724 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1469/2936] 10.222.0.55 () {58 vars in 1224 bytes} [Sat Sep 24 20:00:28 2022] DELETE /identity/v3/users/b1e8669f366f47e0938ed3a4a1024081 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:00:28.660992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Authenticating user token {{(pid=81801) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:00:28.662679 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.fernet_utils [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81801) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:00:28.683355 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4ac8-f057-4a90-a05b-c33ea127ad6c admin admin] Validating token access rules against request {{(pid=81801) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:00:28.697492 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:00:28.775188 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818c3ae70a684477af4280e7bbdcea2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d64e2847698419baef7cbb0bac87311', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81801) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:00:28.775876 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:00:28.776154 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:00:28.776427 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] PATH_INFO: `/v3/projects/f87fa1a153374028977e76e49f5626bb` {{(pid=81801) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:00:28.777790 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87fa1a153374028977e76e49f5626bb)` {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:00:28.779781 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] RBAC: Authorization granted {{(pid=81801) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:00:28.847097 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87fa1a153374028977e76e49f5626bb', 'request_id': 'req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.858507 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87fa1a153374028977e76e49f5626bb', 'request_id': 'req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.864992 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87fa1a153374028977e76e49f5626bb', 'request_id': 'req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c'} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.876990 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] The token cache is being invalidate because project f87fa1a153374028977e76e49f5626bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81801) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:00:28.877542 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.878422 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG keystone.notifications [None req-85e77b95-76cf-4b53-8a2c-2c1c220dd25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81801) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:00:28.880100 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: [pid: 81801|app: 0|req: 1468/2937] 10.222.0.55 () {58 vars in 1233 bytes} [Sat Sep 24 20:00:28 2022] DELETE /identity/v3/projects/f87fa1a153374028977e76e49f5626bb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:00:38.418268 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:00:38.873671 dv-cinder-859191-1-iscsi devstack@keystone.service[81801]: DEBUG dbcounter [-] [81801] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81801) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:03:58.272877 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:03:58.274951 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:03:58.448308 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56a4ad-b2bc-48fa-9390-8889a3102274 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:03:58.516504 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a2247b0344a416788f593199de5e42c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca32d28730cf460ba7618a86aef5a4f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:03:58.517031 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:03:58.517314 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:03:58.517575 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:03:58.519353 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:03:58.531258 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:03:58.532900 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3cc58-f8cc-4ef0-961b-f9fe8ba00882 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:03:58.544519 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1470/2938] 10.222.0.55 () {60 vars in 1365 bytes} [Sat Sep 24 20:03:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:04:08.539089 dv-cinder-859191-1-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}